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MODELING  AND  SIMULATION 


Longuemare  Endorses  Two  Important 
Modeling  and  Simulation  Documents 

RANDY  ZITTEL 


In  1993,  the  Army  established  a 
policy  requiring  that  all  major  pro¬ 
gram  managers  (PM)  plan  their 
use  of  modeling  and  simulation 
(M&S)  throughout  their  pro¬ 
grams.  Because  early  planning  in  a 
program  or  project  is  crucial,  and 
M&S  is  expensive,  the  Army  weighed 
carefully  its  decision  to  require  that  all 
PMs  document  their  planning  efforts 
in  a  specific  Simulation  Support  Plan 
(SSP).  The  positive  potential  of  M&S 
for  accelerating  schedules,  reducing 
cost,  and  improving  quality  was  a  key 
acquisition  reform  initiative  that  Army 
senior  acquisition  managers  wanted  to 
exploit.  Because  of  the  SSP’s  effective¬ 
ness,  in  1996  the  Army  expanded  its 
SSP  requirement  to  include  ACAT  Ill 
and  IV  Army  program  and  product 
managers.  The  intent  was  to  provide 
program  and  product  managers  a 
management  tool  that  would  result  in 
increased  M&S  focus  and  coordina¬ 
tion. 

The  Army  recently  published  an  excel¬ 
lent  pamphlet  to  assist  PMs  in  better 
understanding  the  importance  and 
value  of  the  SSP.  The  Assistant  Secre¬ 
tary  of  the  Army  for  Research,  Devel¬ 
opment,  and  Acquisition  just  released 
Simulation  Support  Plan  Guidelines , 
May  1997,  to  all  acquisition  managers. 

In  a  memorandum  to  all  other  Service 
Acquisition  Executives,  May  2,  1997,  R. 
Noel  Longuemare,  Principal  Deputy 
Under  Secretary  of  Defense  for  Acquisi¬ 
tion  and  Technology,  provided  copies 
of  the  SSP  Guidelines,  and  recommend¬ 
ed  their  use  in  all  Department  of 
Defense  (DoD)  programs. 

The  Simulation  and  Support  Plan  Guide¬ 
lines  will  soon  be  available  on  the 
Assistant  Secretary  of  the  Army  for 


Research,  Development,  and 
Acquisition  (ASARDA)  Home  Page: 

http://www.sarda.army.mil 

To  obtain  a  copy  of  the  Guidelines , 
send  an  E-mail  to:  purdye@sarda. 
army.mil  or  call  (703)  614-5920. 

Another  important  document  relating 
to  current  use  of  DoD  M&S  came  out 
of  the  Office  of  the  Secretary  of 
Defense  (OSD).  Its  “Study  on  the 
Effectiveness  of  Modeling  and  Simula¬ 
tion  in  the  Weapon  System  Acquisition 
Process”  is  an  excellent  overview  of 
current  DoD  use  of  M&S  and  its 
potential  for  the  future.  The  all-encom¬ 
passing  concept  of  SBA  is  introduced 


here,  and  the 

study  provides  a  tremendous  wealth  of 
M&S  use  in  current  programs. 
Longuemare  distributed  the  study 
report  to  all  major  DoD  PMs  in  his 
memorandum  of  March  28,  1997,  as 
an  indication  of  M&S  value,  and  chal¬ 
lenged  each  PM  to  use  M&S  to  the 
maximum  extent  possible,  “to  contin¬ 
uously  reduce  life  cycle  costs.”  This 
capstone  study  is  available  through  the 
Defense  Modeling  and  Simulation 
Home  Page  (http://www.dmso.mil) 
on  the  World  Wide  Web. 

Editor’s  Note:  Zittel  is  a  Professor  of 
Systems  Engineering,  Faculty  Division, 
DSMC. 
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Army  SSP  Guidelines 


THE  UNDER  SECRETARY  OF  DEFENSE 

30  1  O  DEFENSE  PENTAGON 
WASHINGTON,  DC  2030  1-3010 
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ACQUISITION  AND 
TECHNOLOGY 


MAY  2  1997 


MEMORANDUM  FOR  COMPONENT  ACQUISITION  EXECUTIVES 
SUBJECT:  Simulation  Support  Plans 


As  part  of  our  reform  efforts,  I  have  continued  to  encourage  an  increasing  emphasis  on 
the  use  of  modeling  and  simulation  (M&S)  in  our  acquisition  programs  to  reduce  cost  and 
schedule  without  sacrificing  quality  or  performance.  A  key  initiative  in  this  area  is  the  concept 
of  Simulation  Based  Acquisition  (SB  A),  the  employment  of  models  and  simulations  across  all 
functional  areas  throughout  the  entire  acquisition  life  cycle. 

To  foster  the  use  of  M&S  in  acquisition,  and  as  part  of  their  move  towards  SB  A,  the 
Army  recently  published  the  attached  “Simulation  Support  Plan  (SSP)  Guidelines.”  The  SSP,  in 
widespread  use  within  the  Army,  is  a  means  for  developing  and  implementing  an  effective 
strategy  for  the  use  of  M&S  throughout  a  program’s  life  cycle,  and  facilitates  a  Program 
Manager’s  (PM’s)  thinking  through  and  resourcing  a  M&S  program.  The  SSP  Guidelines 
provide  guidance  to  PMs  on  developing  such  a  plan,  and  highlight  the  issues  the  PM  should 
address  in  identifying  how  M&S  can  support  system  development  throughout  the  entire 
acquisition  life  cycle. 

These  guidelines  not  only  provide  a  useful  tool  for  the  PM’s  M&S  tool  kit,  but  are  also  a 
good  example  of  how  the  acquisition  community  can  begin  to  implement  the  concept  of  SB  A.  I 
therefore  encourage  you  to  make  these  Army  guidelines  known  and  available  to  your 
communities  as  one  more  step  towards  developing  and  fielding  our  systems  as  efficiently  as  we 
possibly  can. 


Attachment 


MbftofeeH 

■  Jiiwlts ><?{  »t3. 


R.  Ng^KLonguemare 
Principal  Deputy 


PM  :  SEPTEMBER- OCTOBER  1997 


OSD  l"l&S  Study 


ACQUISITION  AND 
TECHNOLOGY 


PRINCIPAL  DEPUTY  UNDER  SECRETARY  OF  DEFENSE 

30 1 5  DEFENSE  PENTAGON 
WASHINGTON  DC  2030 1-3015 


MAR  28  1997 


jgsoi 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE  OFFICERS 
ACAT I  PROGRAM  MANAGERS 
AC  AT  D  PROGRAM  MANAGERS 

SUBJECT:  Study  on  the  Effectiveness  of  Modeling  and  Simulation  in  the  Weapon  System 
Acquisition  Process 


As  you  continue  to  manage  weapon  system  acquisition  programs  to  the  highest  standards, 
you  have  many  resources  available  to  assist  you  in  building  affordable,  executable  strategies.  I 
urge  you  to  count  this  study  as  one  of  the  many  sources  of  information  you  use  as  you  plan  and 
execute  your  programs. 

This  study  will  aid  you  in  getting  a  perspective  on  the  modeling  and  simulation  (M&S) 
tools  and  technologies  available  that  can  positively  impact  your  program  management.  Many  of 
the  examples  in  this  study  reinforce  what  I  continue  to  emphasize:  emerging  technologies, 
integrated  with  systems  engineering,  can  result  in  cost  avoidance  through  process  efficiencies  in 
your  program’s  life  cycle.  They  can  also  reduce  your  program’s  acquisition  cycle  time. 

Performance,  cost,  and  schedule  are  central  to  the  execution  of  your  programs,  and  you  can 
influence  each  of  these  by  leveraging  some  of  the  M&S  tools  and  technologies  available  in  the 
commercial  and  defense  communities  today.  I  challenge  each  of  you  to  continue  to  insert 
emerging  technology  at  an  affordable  cost  and  to  use  available  processes  to  continuously  reduce 
life  cycle  costs.  You  can  make  a  difference  in  changing  the  way  we  execute  programs. 


Attachment: 
As  stated 


R.  NoeWlonguemare 
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Sanders  is  the  Director,  Test,  System  Engineering  and  Evaluation,  Office  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology). 
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The  national  security  environment  has 
changed  too.  In  the  post-Cold  War 
world,  the  United  States  no  longer 
faces  a  single,  galvanizing  threat  such 
as  the  former  Soviet  Union.  Instead, 
there  is  increased  likelihood  that  U.S. 
forces  will  be  committed  to  limited 
regional  military  actions.  A  statistician 
might  say,  the  mean  value  of  the  single 
greatest  threat  is  considerably  reduced, 
but  the  variance  of  the  collective  threat 
the  country  must  be  prepared  to  meet 
has  increased. 

In  response  to  the  reduced  mean 
value  of  the  threat,  the  United 
States  has  cut  end  strength 
by  about  one-third  from 
1985  levels.  At  the 
same  time,  the 
increase  in  vari¬ 
ance  has  result¬ 
ed  in  a  one- 
third  increase 
in  the  number 
of  U.S.  force 
deployments. 


Procurement  Reductions 

The  overall  U.S.  Department  of  De¬ 
fense  (DoD)  budget  has  been  cut  by 
about  one-third  in  real  dollars  since  its 
peak  in  the  late  1980s.  When  one  con¬ 
siders  that  the  procurement  budget 
changes  by  two  percentage  points  for 
every  percentage  point  the  overall 


DoD  budget  changes  (up  or  down), 
one  realizes  that  a  significant  decrease 
(about  two-thirds)  in  procurement 
funding  has  taken  place. 

Traditionally,  procurement  has  been 
the  most  volatile  component  of  a  DoD 
budget  drawdown  because  — 

•  the  acquisition  of  new  equip¬ 
ment  for  a  smaller  force  struc¬ 
ture  is  viewed  as  unnecessary; 
and 

•  there  is  an  emphasis  on  near- 
term  readiness  and  a  willing¬ 
ness  to  gamble  on  what  consti¬ 
tutes  acceptable  technology. 

The  effect  of  such  procurement 
reductions  on  the  ultimate  user  of  the 
equipment,  i.e.,  the  soldier,  sailor,  air¬ 
man,  or  marine,  must  not  be  underes¬ 
timated.  If  the  issue  of  equipping  the 
military  forces  is  seen  as  a  business 
proposition,  one  can  readily  calculate 
-  by  dividing  the  value  of  all  tangible 
assets  the  DoD  owns  (exclusive  of 
land  and  buildings)  by  the  annual 
reinvestment  in  those  same  assets  — 
that  the  average  item  of  military 
equipment  in  America’s  inventory 
will  have  to  last  54  years!  This  in  a 
world  where  technology  generally  has 
a  half-life  of  from  two  to  10  years,  and 
combat  casualties  are  directly  related 
to  the  quality  of  technology 
employed. 

Since  this  approach  to  the  budget 
defers  long-term  modernization  and  is 
certain  to  have  an  adverse  effect  on 
future  readiness,  it  must  be  interpreted 
as  a  temporary  condition. 
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Need  for  Modernization 
Strategy 

In  view  of  the  overall  federal  budget,  it 
is  only  realistic  to  assume  there  will  be 
continued  pressure  to  limit  increases 
in  defense  investment  spending.  In 
such  a  climate,  it  is  important  to  think 
in  terms  of  a  modernization ,  rather 
than  recapitalization ,  strategy  for 
equipping  US.  forces.  Recapitalization 
suggests  a  one-for-one  replacement  of 
existing  platforms  with  new  platforms 
having  similar  capabilities.  Moderniza¬ 
tion  means  developing  and  fielding 
fewer,  more  capable  systems.  The  key 
question  is:  Can  the  Defense  Depart¬ 
ment  afford  a  modernization-based 
investment  approach?  Technological 
complexity  is  certain  to  increase,  dra¬ 
matically  in  many  instances. 

Norm  Augustine,  Chief  Executive  Offi¬ 
cer  of  Lockheed  Martin,  pointed  out 
some  years  ago  that  the  cost  of  each 
successive  generation  of  fighter  aircraft 
was  increasing  geometrically.  As  a 
result,  although  fighter  aircraft  were 
becoming  more  and  more  deadly,  the 
United  States  could  afford  fewer  and 
fewer  of  them.  Augustine’s  calculation 

-  an  empirical  plot  of  aircraft  unit 
cost  as  a  function  of  deployment  date 

—  was  that  by  some  time  in  the  middle 
of  the  next  century,  the  country  would 
be  able  to  afford  only  one  fearsome, 
sophisticated  aircraft! 

The  geometric  increase  in  cost  results 
because  complex  technologies  become 
more  and  more  interdependent.  For 
example,  a  radio  can  interfere  with  air¬ 
craft  flight  controls  or  have  an  impact 
on  electronic  warfare  equipment.  To 
reduce  radar  signatures,  designers  may 
have  to  shape  an  aircraft  in  a  way  that 
forces  them  to  move  engines, 
weapons,  and  even  the  pilot.  Any  of 
these  actions  can  affect  other  parts  of 
a  system’s  operation,  not  to  mention 
its  producibility  or  logistics  support. 

It  is  essential  to  remember  that  Augus¬ 
tine’s  prediction  is  empirical.  It  is 
based  on  past  experience  and  process¬ 
es  for  handling  the  interaction  of 
increasingly  complex  technologies. 
Industry  and  the  DoD  need  to  share 


responsibility  for  finding  an  alternative 
path  to  fielding  affordable,  modern 
systems. 

Becoming  a  "Smart  Buyer" 

The  DoD  needs  to  become  a  “smart 
buyer,”  in  terms  of  both  what  and  how 
it  buys  equipment.  The  “what”  is  at 
least  as  important  as  the  “how.  ” 

What  to  Buy? 

To  determine  what  it  will  buy,  the  DoD 
is  placing  considerable  emphasis  on  a 
“system-of-systems”  decision-making 
approach,  or  construct.  The  goal  is  to 
select  the  most  cost-effective  mix  of 
individual  systems  for  development 
and  fielding.  Tradeoffs  between  on¬ 
board  and  off-board  capabilities  are 
being  considered,  and  alternative  sys¬ 
tems  are  being  evaluated  under  simu¬ 
lated  combat  conditions. 

Recently,  the  Heavy  Bomber  Study 
looked  at  the  adequacy  of  the  planned 
bomber  force  in  the  context  of  a  two- 
major-region,  contingency  scenario. 
The  Strategic  Airlift  Force  Mix  Analysis 
and  Tactical  Utility  Analysis  were  used 
to  evaluate  the  cost  effectiveness  of 
various  mixes  of  C-17  aircraft  and 
nondevelopmental  airlift  platforms  to 
perform  airlift  missions  in  support  of 
various  contingency  operations.  A  sim¬ 
ilar  study  is  currently  in  progress  to 
evaluate  the  mix  of  accurately  guided 
weapons  the  Department  is  procuring. 

A  hierarchy  of  models  and  simulations 
is  being  used  to  support  these  studies 
and  to  help  make  the  what-to-buy 
decisions.  First,  at  the  engagement  or 
system  level,  the  system  effectiveness 
against  an  adversary  system  is  evaluat¬ 
ed.  Later,  at  the  mission/battle  or 
force-on-force  level,  the  ability  of  a 
multiple  platform  force  package  to 
perform  a  specific  mission  is  assessed. 
Finally,  in  theater-  or  campaign-level 
simulations,  the  conflict  outcomes  are 
determined  for  a  total  package  of  Joint 
and  Combined  forces. 

Extensive  use  of  constructive  models 
for  these  system-of-systems  evalua¬ 
tions  is  anticipated.  Eventually,  there 
will  be  much  greater  use  of  virtual  pro¬ 


totypes  operated  on  synthetic  battle¬ 
fields.  Without  question,  the  DoD  is 
moving  toward  greater  use  of  simula¬ 
tion-based  system  evaluations. 

The  Department’s  what-to-buy  deci¬ 
sions  are  also  being  driven  by  life¬ 
cycle-cost-performance  trades  where 
cost  is  an  independent  variable.  Gone 
are  the  days  when  performance  was 
paramount,  and  cost  took  a  back  seat 
and  was  treated  as  a  dependent  vari¬ 
able.  Life-cycle-cost-performance 
trades  require  evaluation  of  alternative 
designs  and  concepts.  Computer 
modeling  and  simulation,  including 
virtual  prototypes,  are  needed  to 
assess  the  performance  of  alternative 
designs  in  a  simulated  combat  envi¬ 
ronment.  They  are  also  needed  to 
examine  the  logistics,  manufacturing, 
and  producibility  implications  of  alter¬ 
native  designs,  and  the  cost  and 
schedule  impacts  of  pursuing  alterna¬ 
tive  designs. 

How  to  Buy? 

The  DoD  must  also  change  how  it 
buys.  The  Department  has  worked  to 
find  the  best  methods  for  reengineer¬ 
ing  its  processes.  In  May  1995  the  Sec¬ 
retary  of  Defense  directed  a  “funda¬ 
mental  change  in  the  way  we  acquire 
goods  and  services”  and  mandated  that 
the  concepts  of  Integrated  Product  and 
Process  Development  (IPPD)  and  Inte¬ 
grated  Product  Teams  (IPT)  “be  applied 
throughout  the  acquisition  process  to 
the  maximum  extent  possible.” 

The  DoD  defines  IPPD  as  “a  manage¬ 
ment  process  that  integrates  all  activi¬ 
ties  from  product  concept  through 
production/field  support,  using  a  mul¬ 
tifunctional  team,  to  simultaneously 
optimize  the  product  and  its  manufac¬ 
turing  and  sustainment  processes  to 
meet  cost  and  performance  objec¬ 
tives.”  An  outgrowth  of  concurrent 
engineering  practices,  the  IPPD 
process  reflects  a  systems  engineering 
approach  that  has  incorporated  sound 
business  practices  and  commonsense 
decision  making.  Fundamental  to  the 
successful  implementation  of  the  IPPD 
concept  will  be  the  willingness  of 
organizations  to  undertake  and  experi- 
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ence  profound  changes  in  their  cul¬ 
tures  and  past  practices. 

To  reduce  the  costs  associated  with 
the  integration  of  complex  systems,  it 
will  be  essential  for  the  functional 
members  of  an  IPT  (e.g,  design  engi¬ 
neering,  manufacturing,  logistics, 
product  support)  to  understand  the 
concerns  of  their  counterparts  and  to 
identify  a  program’s  technical  chal¬ 
lenges  as  early  as  possible.  Tools  avail¬ 
able  to  an  IPT  include  standard, 
relatively  inexpensive  computer  equip¬ 
ment,  virtual  prototypes,  and  simula¬ 
tions.  Such  resources  can  aid  in  the 
development  of  a  shared  vision  of  the 
proposed  system  and  provide  a  means 
for  understanding  the  complex  inter¬ 
actions  among  the  configuration  items 
in  the  system  design. 

The  real  power  of  a  computer-based 
modeling  and  simulation  system  lies 
in  the  connection  and  coordination 
between  the  tools  and  the  functional 
users.  In  addition  to  increasing  the 
effectiveness  of  the  design  and  manu¬ 
facturing  functional  specialists,  the 
product  support  members  of  the  team 
(e.g.,  testers,  logisticians,  and  main¬ 
tained)  will  benefit  as  well. 

Simulation  Based  Acquisition 

The  DoD  envisions  an  acquisition 
process  supported  by  the  robust,  collab¬ 
orative  use  of  simulation  technology  that 
is  integrated  across  acquisition  phases 
and  programs.  The  objectives  of  Simula¬ 
tion  Based  Acquisition  (SBA)  are  to  — 

•reduce  the  time,  resources,  and 
risk  associated  with  the  acquisi¬ 
tion  process; 

•increase  the  quality,  military 
utility,  and  supportability  of  sys¬ 
tems  developed  and  fielded; 
and 

•  enable  IPPD  from  requirements 
definition  and  initial  concept 
development  through  testing, 
manufacturing,  and  fielding. 

Substantial  evidence  has  already  accu¬ 
mulated  regarding  the  value  of  a  simu- 
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lation-based  approach  to  acquisition. 
Both  commercial  and  military  pro¬ 
grams  provide  pervasive  evidence  of 
tangible  results  that  can  be  measured 
in  terms  of  improvements  in  cost , 
schedule ,  productivity ;  and  quality/per¬ 
formance. 


Cost 

The  LPD-17  program  saved  $6  million 
in  design  costs  through  the  use  of  new 
modeling  and  simulation  tools.  At  the 
same  time,  it  was  able  to  eliminate  100 
tons  in  topside  weight,  a  design 
change  expected  to  result  in  greatly 
improved  performance.  In  the  Joint 
Strike  Fighter  program,  it  is  projected 
that  virtual  manufacturing  techniques 
may  save  as  much  as  3  percent  of  the 
program’s  estimated  life-cycle  cost, 
which  could  be  $5  billion. 


Schedule 

The  use  of  modeling  and  simulation 
tools  and  processes  by  the  “big  three” 
auto  manufacturers  has  reduced  the 
time  from  concept  approval  to  produc¬ 
tion  from  five  to  three  years,  and  sig¬ 
nificant  further  schedule  reductions 


are  anticipated.  Separately,  Electric 
Boat™  reports  it  has  been  able  to  halve 
the  time  required  for  submarine  devel¬ 
opment,  from  14  to  seven  years. 

Productivity 

Productivity  is  also  affected  by  the 
increased  use  of  modeling  and  simula¬ 
tion.  The  required  level  of  effort  (per¬ 
son  years)  is  often  less,  and  fewer 
workers  may  be  needed.  Costly  inter¬ 
mediate  steps  (e.g.,  mockups,  re¬ 
designs,  and  engineering  changes)  can 
frequently  be  avoided,  there  is  reduced 
scrap,  and  less  manufacturing  floor 
space  is  required  when  modeling  and 
simulation  are  used. 

It  took  38  Sikorsky  draftsmen  approxi¬ 
mately  six  months  to  develop  working 
drawings  of  the  CH-53E  Super  Stal¬ 
lion’s  outside  contours.  In  contrast, 
using  modeling  and  simulation  one 
engineer  was  able  to  accomplish  the 
same  task  for  the  Commanche  heli¬ 
copter  in  just  one  month.  In  another 
instance,  14  engineers  at  the  Tank  and 
Automotive  Research  and  Develop¬ 
ment  Center  designed  a  new,  low-sil¬ 
houette  tank  prototype  in  only  16 
months,  a  task  that  would  have 
required  approximately  55  engineers 
and  three  years  with  more  traditional 
methods. 

Quality/Performance 
The  positive  impact  of  modeling  and 
simulation  on  quality  and  perfor¬ 
mance  can  be  seen  in  a  number  of 
areas,  e.g.,  the  proper  assembly  of 
products  and  systems,  fewer  instances 
where  rework  is  needed,  a  reduced 
parts  count,  and  the  opportunity  for 
early  design  evaluation  prior  to  further 
design  efforts. 

For  example,  Northrop’s  use  of  CAD 
[computer-aided  design]  systems  led 
to  a  first-time,  error-free,  physical 
mockup  of  many  sections  of  the  B-2 
aircraft.  In  the  case  of  the  Navy’s  Next 
Generation  New  Attack  Submarine, 
new  modeling  and  simulation  tools 
helped  reduce  the  standards  parts  list 
to  about  16,000  items  from  the  95,000 
items  listed  for  the  earlier  Seawolf- 
class  submarine. 
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Embracing  This  Approach  — 
What  is  Needed? 

It  is  clear  that  IPPD,  backed  by  a 
strong  commitment  to  computer- 
based  modeling  and  simulation  tools, 
provides  a  dominant  and  competitive 
edge  in  the  commercial  marketplace 
and  a  distinct  war  fighting  advantage 
on  the  battlefield.  It  provides  an  alter¬ 
nate  path  for  getting  to  market  first,  at 
lower  cost.  In  the  process,  quality  is 
improved.  The  underlying  technology 
is  widely  available,  and  market  forces 
are  driving  industry  toward  SBA.  So 
what  is  needed  to  fully  embrace  this 
approach? 

SBA  is  comprised  of  three  principal 
components.  The  first  is  an  advanced 
systems  engineering  environment  that 
uses  formal  methods  and  automation 
to  support  efficient  design  synthesis, 
capture,  and  assessment,  as  well  as 
other  complex  life-cycle  activities.  The 
SBA  engineering  environment  pro¬ 
vides  a  means  for  executing  a  process 
that  can  be  extended,  tailored,  and 
repeated.  The  process  results  in  the 
creation  of  reusable  design  reposito¬ 
ries  and  products  that  can  be  reengi¬ 
neered.  The  potential  gains  from  the 
use  of  such  an  advanced  SBA  envi¬ 
ronment  will  not  be  realized  until  the 
engineering  process,  as  well  as  its 
people  and  organizations,  also 
evolve. 

The  second  component  is  a  refined  sys¬ 
tem  acquisition  process  that  takes  advan¬ 
tage  of  the  SBA  systems  engineering 
environment  capabilities.  The  third 
component  is  a  culture  that  has 
evolved  to  a  point  where  enterprise¬ 
wide  cooperation  is  the  rule,  and  indi¬ 
vidual  technical  contributions  and 
innovations  are  encouraged  and  man¬ 
aged  efficiendy. 

SBA  is  not  an  incremental  step  beyond 
current  system  engineering  methods 
and  tools.  Instead,  it  represents  a 
major  paradigm  shift  toward  a  compre¬ 
hensive,  integrated  environment  that 
addresses  the  entire  system  develop¬ 
ment  life  cycle  and  the  spectrum 
of  engineering  and  management 
domains. 


It  is  clear  that  IPPD, 
backed  by  a  strong 
commitment  to 
computer-based 
modeling  and 
simulation  tools, 
provides  a  dominant 
and  competitive  edge 
in  the  commercial 
marketplace  and  a 
distinct  warfighting 
advantage  on  the 
battlefield. 


The  benefits  from  the  SBA  process  will 
be  realized  not  only  as  time  and  cost 
savings  within  individual  programs, 
but  also  as  cost  savings  when  a  pro¬ 
gram  makes  use  of  design  repositories 
and  reengineered  tools  and  products 
from  other  programs. 

Cross-Program  Use  of  Data, 
Tools,  and  Techniques 

Modeling  and  simulation  tools,  as 
enablers  for  IPPD  development,  are 
already  being  applied  successfully  to 
reduce  development  time  and  life- 
cycle  costs  in  a  range  of  ongoing 
acquisition  programs.  The  issue  is  no 
longer  whether  extensive  use  of  mod¬ 
eling  and  simulation  tools  has  merit, 
but  rather  how  to  develop  and  apply  a 
new  acquisition  process  in  a  deliberate 
and  coordinated  manner  that  uses 
these  tools  to  maximum  advantage 
and  achieves  even  more  dramatic 
reductions  in  cost,  schedule,  and  risk. 

The  challenge  for  acquisition  reform  is 
to  provide  the  catalyst  that  will  expand 
the  growing  successful  use  of  model¬ 
ing  and  simulation  tools  beyond  verti¬ 
cal  applications  within  individual  pro¬ 
grams.  If  this  is  accomplished,  even 


more  significant  benefits  will  be  real¬ 
ized  through  the  shared  use  of  data, 
tools,  and  techniques  by  government 
and  industry.  Unambiguous  commu¬ 
nication  is  required  to  achieve  full 
application  of  the  IPPD  and  IPT 
processes;  such  communication  can 
serve  as  the  catalyst  that  encourages  a 
new  acquisition  culture  to  use  these 
powerful  new  tools  and  processes. 

Partnership 

The  challenge  is  clear:  The  trend 
toward  geometrically  escalating  costs 
in  successive  generations  of  defense 
equipment  must  be  reversed.  Limiting 
the  sophistication,  and  therefore  the 
capability,  of  future  systems  is  not  a 
realistic  option.  The  task  is  to  field 
increasingly  complex  technologies  at  a 
more  affordable  cost,  in  less  time. 

This  will  require  a  team  effort  by 
industry  and  the  DoD  to  field  a  supe¬ 
rior  capability,  affordably  and  in  less 
time  than  potential  adversaries.  Indus¬ 
try  needs  to  use  the  latest  information 
technologies  to  upgrade  its  integrated 
product  capabilities.  The  DoD  needs 
to  become  a  smarter  buyer.  Together, 
industry  and  government  must  ensure 
that  the  acquisition  management  cul¬ 
ture  evolves  to  — 

•  take  advantage  of  IPPD  ap¬ 
proaches  that  stress  the  need 
for  a  shared  vision  and  continu¬ 
ous  insight  to  ensure  that  quali¬ 
ty  is  built  into  programs  from 
the  start; 

•  emphasize  prevention  over 
cures  by  using  virtual  proto¬ 
types  and  simulations  to  identi¬ 
fy  and  resolve  problems  early; 
and 

•focus  on  overall  program  suc¬ 
cess,  not  functional  area  perfor¬ 
mance. 

The  appropriate  vehicle  for  meeting 
this  challenge  is  SBA  a  method  which 
combines  a  new  process,  new  tools, 
and  a  new  culture  to  develop  a  strong 
collaborative  partnership  between  gov¬ 
ernment  and  industry 
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Dr.  Patricia  Sanders  is  the  Director,  Test,  Systems  Engineering  and  Evaluation  (DTSE&E)  for  the 
Department  of  Defense  (DoD)  where  she  is  responsible  for  ensuring  the  effective  integration  of  all 
engineering  disciplines  into  the  system  acquisition  process.  These  include  design,  production,  man¬ 
ufacturing  and  quality,  acquisition  logistics,  modeling  and  simulation,  and  software  engineering,  with 
emphasis  on  test  and  evaluation  as  the  feedback  loop.  She  is  also  responsible  for  oversight  of  the 
Department  of  Defense’s  Major  Range  and  Test  Facility  Base  (MRTFB)  and  the  development  of  test 
resources  such  as  instrumentation,  targets,  and  other  threat  simulators.  The  MRTFB  comprises  more 
than  50  percent  of  the  DoD  land  resources,  represents  a  capital  investment  of  more  than  $25  billion, 
and  employs  approximately  47,000  government  and  contractor  personnel.  Sanders  chairs  the  Defense 
Test  and  Training  Steering  Group,  the  Systems  Engineering  Steering  Group,  and  the  Acquisition  Council 
on  Modeling  and  Simulation.  She  reports  directly  to  the  Principal  Deputy  Under  Secretary  of  Defense 
for  Acquisition  and  Technology. 

Sanders  has  over  22  years  of  experience  in  the  Department  of  Defense  with  particular  emphasis  in  the 
areas  of  test  and  evaluation,  modeling  and  simulation,  resource  allocation,  and  strategic  planning.  Prior 
positions  within  the  Office  of  the  Secretary  of  Defense  included  serving  as  the  Deputy  Director  for  Test 
Facilities  and  Resources,  the  Director  of  Land  Forces  in  the  Office  of  the  Assistant  Secretary  of  Defense 
for  Program  Analysis  and  Evaluation,  and  as  a  Staff  Specialist  for  the  Director  of  Operational  Test  and 
Evaluation.  Other  assignments  have  included  serving  as  Deputy  Director  for  Analysis,  United  States 
Space  Command;  Science  Advisor  to  the  Command,  Control,  Communications,  and  Countermeasures 
Joint  Test  Force;  and  Chief  of  Modeling  and  Simulation  and  Technical  Advisor  to  the  Electronics  Systems 
Division  at  the  Air  Force  Operational  Test  and  Evaluation  Center.  Her  government  career  was  preceded 
by  university  faculty  positions. 

Sanders  received  her  doctorate  in  mathematics  in  1972  as  a  National  Science  Foundation  Fellow  at 
Wayne  State  University  and  is  a  1992  graduate  of  the  Senior  Executive  Fellow  Program,  John  F 
Kennedy  School  of  Government,  Harvard  University.  She  is  a  member  of  the  Senior  Advisory  Board 
and  a  past  President  of  the  International  Test  and  Evaluation  Association  (ITEA),  a  Fellow  of  the  Ameri¬ 
can  Institute  of  Aeronautics  and  Astronautics,  and  a  member  of  the  Board  of  Directors  of  the  Military 
Operations  Research  Society. 
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Modeling  and  Simulation  (M&S)  Use  in 
the  Army  Acquisition  Process 

Shift  to  Simulation  Based  Acquisition  Recognizes 
N&S  As  Tremendous  Opportunity  for  PNs 


DR.  HERBERT  K.  FALUN,  JR. 


Anew  paradigm  is  emerging  in 
the  Army  regarding  the  use  of 
Modeling  and  Simulation 
(M&S)  in  the  acquisition 
process.  This  new  paradigm  is 
Simulation  Based  Acquisition  (SBA). 

Under  the  old  school  of  thought, 

M&S  was  regarded  as  just  another  tool 
to  be  used  in  the  design  of  a  weapon 
system.  The  shift  to  SBA  recognizes 
that  M&S  represents  tremendous 
opportunity  for  the  program  manager 
(PM)  and  is  more  than  just  a  tool  to  be 
taken  for  granted.  PMs  today  recog¬ 
nize  that  M&S  must  be  managed  as  a 
resource  in  order  to  achieve  the  bene¬ 
fits  inherent  in  the  use  of  M&S 
throughout  the  acquisition  process.  In 
order  to  capitalize  on  these  benefits, 

PMs  must  be  savvy  in  two  critical 
areas: 

•What  is  SBA? 

•Just  how  it  is  implemented? 

The  use  of  M&S  in  the  acquisition 
process  is  nothing  new  to  the  Army,  use  of  M&S  throughout  all  phases  of  •  Continuous  evaluation  of  sys- 
What  is  new  is  the  increasing  availabil-  the  acquisition  cycle,  to  ultimately  tem  development, 
lty  and  power  of  M&S  tools  and  the  deliver  fielded  systems  within  imposed  •  Rapid  evaluation  of  concept 
decreased  availability  of  resources  for  budget  constraints.  When  properly  design, 
weapon  system  development.  These  incorporated  into  a  program,  SBA  •  Reduce  and  delay  need  for 
two  occurrences  have  served  as  a  fore-  yields  the  following  benefits,  which  act  physical  prototype, 

ing  function,  steering  the  acquisition  to  reduce  risk  in  cost,  schedule,  and  -Facilitate  continuous  user  par- 

community  into  better  integrating  the  performance:  ticipation  in  development 


Figure  1  What  is  Simulation  Based  Acquisition? 
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process. 

•  Efficient  development/ evalua¬ 
tion  of  manufacturing  plans. 

•  Reuse  of  system  software  and 
hardware  in  training  simula¬ 
tors. 

•Ability  to  test  proposed  system 
at  sub-component,  component, 
and  system  level. 
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What  Is  Simulation  Based 
Acquisition? 

SBA  is  a  concept  for  efficiently  manag¬ 
ing  M&S  as  a  resource  to  be  exploited 
by  the  PM  in  the  effort  to  accomplish 
acquisition  objectives.  As  we  shift 
toward  more  efficient  and  effective  use 
of  M&S,  the  abandonment  of  “stove- 
piping”  techniques  for  employing 
M&S  must  become  a  reality.  The 
boundaries  imposed  by  the  acquisi¬ 
tion  phases  and  milestones  are  no 
longer  constraints  to  those  who  opti¬ 
mize  the  use  of  M&S.  Re-use  of  M&S 
for  multiple  functions  and  linking  dif¬ 
ferent  models  and  simulations  across 
all  phases  of  acquisition  is  a  powerful 
concept  with  benefits  that  are  current¬ 
ly  being  realized.  SBA  is  characterized 
by  a  more  flexible  and  integrated 
approach  to  using  M&S  in  the  acquisi¬ 
tion  process. 

As  depicted  in  Figure  1,  the  utility  of 
the  SBA  concept  to  the  PM  lies  in  the 
notion  that  M&S  developed  for  use  in 
a  functional  area  can  serve  in  a  similar 
capacity  to  accomplish  tasks  in  each  of 
the  phases,  from  concept  exploration 


to  operations  and  support  (O&S). 
Usually  the  M&S  evolves  as  the  pro¬ 
gram  progresses  until  a  full  suite  of 
models  evolves,  which  represents  the 
entire  weapon  system.  Linking  models 
together  using  one  model’s  output 
data  as  input  data  for  another  model 
generates  efficiencies  for  the  PM  that 
allow  reductions  in  cost  and  schedule. 

Identifying  how  M&S  can  be  used 
across  the  acquisition  phases  and  in 
the  various  functional  areas  represents 
the  first  step  in  developing  the  Simula¬ 
tion  Support  Strategy.  This  strategy 
focuses  on  the  appropriate  mix,  type, 
and  fidelity  of  M&S  tools.  One  of  the 
largest  barriers  to  the  effective  execu¬ 
tion  of  the  Simulation  Support  Strate¬ 
gy  in  the  Army  was  the  inability  to 
clearly  articulate  M&S  requirements 
to  those  responsible  for  the  actual 
development  of  M&S.  To  rectify  this 
problem,  the  Simulation  Support  Plan 
(SSP)  Guidelines,  which  are  discussed 
later  in  this  article,  were  introduced. 
These  guidelines  require  Army  PMs  to 
craft  a  Simulation  Support  Strategy 
and  package  this  strategy  in  a  format 


that  clearly  identifies  and  communi¬ 
cates  M&S  requirements  to  the  mod¬ 
eling  community  -  a  format  referred 
to  as  the  “M&S  Tool  Kit.” 

Figure  2  illustrates  the  mapping  of 
M&S  tools  to  M&S  requirements. 
This  is  the  essence  of  the  SSP. 

How  To  Incorporate  SBA 

The  SSP  is  the  implementing  tool  the 
Army  uses  to  employ  M&S  in  the 
most  effective  and  efficient  manner 
possible.  This  construct  was  initiated 
in  1993  by  the  Military  Deputy  to  the 
Army  Acquisition  Executive.  In  1996, 
OSD  implemented  a  policy  that  re¬ 
quired  all  ACAT  I  and  II  programs  to 
coordinate  their  SSPs  with  various 
Army  activities  and  include  an  M&S 
strategy  summary  in  the  Acquisition 
Strategy  Report.  The  SSP  Guidelines, 
published  and  distributed  in  May 
1997,  further  supplemented  this  guid¬ 
ance.  Additionally,  in  his  May  2,  1997, 
memorandum,  the  Principal  Deputy 
to  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology  encour¬ 
aged  all  the  Services  to  use  the  Army’s 
SSP  Guidelines  as  a  model  for  PMs  to 
organize  their  respective  M&S  strate¬ 
gies  and  implement  SBA 

The  intent  of  the  SSP  is  to  provide  a 
management  tool  that  assists  the  PM 
in  thinking  through  M&S  require¬ 
ments  for  the  acquisition  program. 
Additionally,  the  SSP  provides  visibility 
of  M&S  capabilities  to  not  only  the 
PM  and  supporting  communities,  but 
to  other  system  PMs  and  programs  in 
other  Services.  Such  visibility  pro¬ 
motes  possible  re-use  of  M&S. 

The  SSP,  when  properly  crafted,  con¬ 
veys  more  than  just  what  M&S  is 
being  used  to  support  the  program.  It 
provides  a  road  map  to  the  PM,  and 
the  acquisition  community,  which 
indicates  what  types  of  M&S  are 
required  and  when  the  M&S  is  need¬ 
ed  to  meet  program  objectives.  The 
SSP  is  the  vehicle  that  allows  the  PM  to 
thoroughly  integrate  the  use  of  M&S 
into  the  acquisition  strategy.  Figure  3 
shows  how  the  SSP  road  map  ties  in 
directly  with  the  acquisition  strategy. 


Figure  2.  Generic  Top-Down  Level  Representation  of  an  SSP 
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As  indicated  in  the  figure,  the  use  of 
M&S  in  the  functional  areas  occurs 
across  all  of  the  acquisition  phases. 

Just  as  the  PM  develops  an  acquisition 
strategy  for  the  desired  system,  so  too 
must  the  PM  develop  a  strategy  for 
M&S.  The  SSP  indicates  not  only 
what  M&S  is  required  to  support  sys¬ 
tem  acquisition,  but  also  when  the 
M&S  should  be  available  for  use,  and 
when  and  how  verification,  validation, 
and  accreditation  (W&A)  will  be  per¬ 
formed. 

The  concept  of  managing  M&S  as  a 
resource  is  not  always  readily  obvious. 
Typically,  tools  are  not  thought  of  as 
requiring  management  attention. 
Because  of  the  tremendous  capability 
of  M&S  to  reduce  cost  and  schedule 
as  well  as  mitigate  associated  risk, 
the  PM  who  does  not  actively  manage 
M&S  activities  risks  fielding  a  sys¬ 
tem  that  is  over  budget  and  behind 
schedule. 


A  helpful  analogy  in  understanding 
why  it  is  important  to  manage  M&S 
tools  is  to  think  in  terms  of  a  do-it- 
yourself  home  project  (such  as  build¬ 
ing  a  set  of  storage  cabinets).  Anyone 
who  has  ever  embarked  on  such  a  ven¬ 
ture  has  a  full  appreciation  of  why  the 
proper  tools  are  so  important.  With 
the  right  tool,  a  daunting  task  can 
become  easy.  Prior  to  starting  that 
home  project,  a  set  of  plans  is  needed 
along  with  a  list  of  required  materials. 
The  mistake  many  first  time  do-it- 
yourselfer’s  make  is  not  realizing  it  is 
just  as  important  to  have  a  plan  for 
how  to  use  the  needed  tools  and 
when  to  have  them  available.  Because 
this  is  so  often  overlooked,  time  is  fre¬ 
quently  lost  because  the  right  type  of 
tool  was  unavailable  when  needed. 
Work  has  to  be  interrupted  to  fetch 
the  needed  tool.  In  some  cases,  if 
prior  thought  had  gone  into  identify¬ 
ing  the  best  type  of  tool  for  a  job  (a 
sliding  compound  miter  saw  instead 
of  a  circular  saw  for  instance),  the  job 


could  have  been  accomplished  in  not 
only  less  time,  but  also  with  less  effort 
and  cost. 

The  same  holds  true  for  M&S.  A  PM 
who  takes  the  time  to  identify  the  best 
set  of  M&S  tools  that  can  be  used  to 
accomplish  needed  tasks  will  ultimate¬ 
ly  field  a  better  product.  M&S  can  be 
used  to  augment  the  systems  develop¬ 
ers’  capabilities.  M&S  provides  the 
means  for  conducting  “what  if”  drills 
when  exploring  new  concepts  or 
stressing  a  system’s  performance.  It 
can  also  be  used  to  identify  design 
flaws,  thus  reducing  and  delaying  the 
need  for  a  physical  prototype.  M&S 
facilitates  user  participation  in  the 
design  process  so  that  the  fielded  sys¬ 
tem  has  increased  quality,  military  util¬ 
ity,  and  supportability.  A  PM  who 
develops  and  implements  a  well 
thought-out  M&S  strategy  will  end  up 
with  an  improved  acquisition  strategy 
as  well  as  a  superior  product  in  the 
field. 
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U.S.  AIR  FORCE  NEWS  RELEASE 

Air  Force  Space  Command 
Establishes  First  Space  Battlelab 

New  Space  Battlelab  Will  Employ  Modeling  and 
Simulation  in  an  Operational  Environment 

CAPT.  CLIFF  D.  OZMUN,  U.S.  AIR  FORCE 


FALCON  AFB,  COLO.  A  new  era 
in  warfighting  was  born  here 
June  30  with  the  activation  of 
the  Space  Battlelab,  an  organiza¬ 
tion  dedicated  to  innovative 
space  operations  and  concepts. 

The  flag  of  one  of  the  Air  Force's 
newest  organizations  was  unfurled  at 
the  activation  ceremony  that  was 
observed  by  Gen.  Howell  M.  Estes  III, 
Commander  in  Chief  of  North  Ameri¬ 
can  Aerospace  Defense  Command, 
U.S.  Space  Command,  and  Comman¬ 
der  of  Air  Force  Space  Command;  and 
Col.  Jeff  Wenzel,  the  battlelab’s  com¬ 
mander. 

“The  Space  Battlelab  will  be  develop¬ 
ing  and  examining  new  ways  to  make 
space  an  integral  part,  not  only  of 
what  our  operational  warfighters  do, 
but  our  logisticians,  our  communica¬ 
tors,  our  intelligence  agencies,  and 
eventually  the  American  public  at 
large,”  said  Estes.  Citing  the  Global 
Positioning  System  as  an  example, 
Estes  said  the  concepts  the  Space  Bat¬ 
tlelab  develops  may  result  in  spin-off 
technologies  that  will  have  application 
to  the  everyday  lives  of  all  American 
citizens,  long  after  the  concepts  begin 
to  serve  the  military’s  needs. 

The  post-Cold  War  environment  creat¬ 
ed  several  new  realities  for  the  military, 
realities  this  battlelab  was  created  to 
address.  Foremost  among  those  reali¬ 
ties  was  the  fact  that  Defense  Depart¬ 
ment  budgets  and  personnel  numbers 
were  significantly  reduced.  Combined 
with  this  was  the  rapid  advancement 
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of  technology  development  and  the 
challenges  this  advancement  poses  for 
upgrading  military  capabilities.  And 
finally  commercial  business  ventures 
have  now  replaced  the  military  as  dri¬ 
vers  of  many  high-technology  markets. 

“The  nature  of  the  combat  environ¬ 
ment  today  is  changing,”  said  Wenzel. 
“Technology  is  moving  faster  than  it 
ever  has  before.  We  don’t  know  if 
we’re  applying  technology  that  our 
country  develops  to  our  warfighting 
the  way  that  we  could  or  should.”  The 
military  is  having  to  fight  in  new,  non- 
traditional  environments  such  as 
Somalia,  Bosnia,  and  Haiti. 

“So  we  need  to  be  able  to  change  and 
do  things  differently  than  the  way 
we’ve  done  them  before,”  said  Wenzel. 
The  Space  Battlelab  facilitates  ideas 
and  innovation,  the  kind  of  innovation 
that  led  to  many  of  the  Air  Force’s  his¬ 
torical  successes. 

Wenzel  said  the  battlelab  is  not  a  “lab¬ 
oratory,”  in  the  classic  sense.  There  are 
no  test  tubes,  beakers  or  Bunsen  burn¬ 
ers.  “I’d  call  us  an  innovation  cell,”  he 
said. 

“As  we  stand  here  today,  anticipating 
the  turn  of  the  century,  on  the  brink 
of  an  evolving  air  and  space  force  to  a 
space  and  air  force,  activating  the  first 
battlelab  for  space,  we  are  indeed  liv¬ 
ing  in  interesting  times,”  said  Estes.  In 
fact,  Estes  said,  many  historic  parallels 
exist  between  the  birth  of  aviation  and 
the  birth  of  the  space  battlelab.  The 
Wright  brothers  had  a  dream,  a  con¬ 
cept  which  became  a  reality  and  the 
foundation  for  the  U.S.  Air  Force. 

“These  men  were  visionaries,  visionar¬ 
ies  whose  concepts  resulted  in  techno¬ 
logical  development  which  changed 
the  course  of  human  events,”  Estes 
said.  “The  need  for  our  air  and  space 
forces  are  evolving  and  moving  for¬ 
ward  into  the  future  at  a  very,  very  fast 
pace.” 

The  Space  Battlelab  is  one  of  six  bat- 
tlelabs  founded  by  the  Air  Force 
whose  missions  are  to  advance  the  Air 


“As  the  battlelabs  begin 
to  work  together,  the 
synergistic  effects  will 
lead  ns  all  into  the  next _ 
century  and  beyond, 
not  only  changing  the 
nature  of  conflict  but 
more  importantly, 
providing  new  ways  to 
make  the  world  a  safer 
place  for  all  who 
inhabit  the  Earth.  ” 


Force  Core  Competencies  of:  Air  and 
Space  Superiority,  Global  Attack,  Preci¬ 
sion  Engagement,  Information  Superi¬ 
ority,  Rapid  Global  Mobility,  and  Agile 
Combat  Support.  The  battlelabs  will 
rely  on  field  innovation  to  identify 
ways  to  advance  these  core  competen¬ 
cies. 

“As  the  battlelabs  begin  to  work 
together,  the  synergistic  effects  will 
lead  us  all  into  the  next  century  and 
beyond,  not  only  changing  the  nature 
of  conflict  but  more  importantly,  pro¬ 
viding  new  ways  to  make  the  world  a 
safer  place  for  all  who  inhabit  the 
Earth,”  Estes  said. 

“We  are  an  air  and  space  force  that 
embraces  change  in  technology,  and 
the  Space  Battlelab  will  lead  the  way  in 
innovations  that  haven’t  been  consid¬ 
ered  yet,”  said  Estes.  The  Space  Batde- 
lab  will  be  small  and  will  focus  on 
innovation  for  space-related  Air  Force 
Operations.  It  will  employ  field  inge¬ 
nuity,  modeling  and  simulation,  and 
existing  capabilities  in  an  operational 
environment  in  order  to  accomplish 
the  Air  Force  mission.  “The  Space  Bat¬ 
tlelab  offers  our  command  and  the  air 
and  space  forces  at  large  the  opportu¬ 
nity  to  consider  concepts  that  will  not 
only  further  integrate  space  into  our 


land,  sea  and  air  forces,  but  go  beyond 
traditional  methods  of  power  protec¬ 
tion,  and  most  importantly,  further 
develop  space  itself,”  Estes  said. 

The  Space  Battlelab  will  report  directly 
to  the  Space  Warfare  Center  here, 
another  cutting-edge  organization 
dedicated  to  marrying  space-based 
capabilities  with  warfighter  needs. 

The  battlelab  will  develop  concepts 
and  rapidly  evaluate  their  potential. 
“We’re  going  to  take  ideas  from  all 
over  the  Air  Force  and  Space  Com¬ 
mand,”  said  Wenzel.  He  adds  that 
when  the  battlelab  gets  an  idea  that 
will  help  the  Air  Force  execute  a  com¬ 
bat  mission  more  efficiently,  the  con¬ 
cept  will  be  tested  and  evaluated.  “And 
then  we’ll  run  with  it.” 

To  illustrate  the  importance  of  these 
battlelabs,  successfully  demonstrated 
battlelab  initiatives  may  result  in 
changes  to  Air  Force  doctrine,  new 
statements  of  combat  mission  needs, 
new  Air  Force  requirements,  repro¬ 
gramming  of  funds,  demonstrations  of 
advanced  technology  concepts,  or 
changes  to  ongoing  or  future  acquisi¬ 
tions. 

“This,  of  course,  is  the  50th  anniver¬ 
sary  year  of  our  Air  Force.  And  we  can 
now  see  the  beginnings  of  the  space 
and  air  force  of  the  future,”  said  Estes. 
“As  we  embark  on  the  next  50  years, 
the  Space  Battlelab  will  play  a  pivotal 
role  in  developing  and  evaluating  con¬ 
cepts  that  will  chart  the  future  of  mili¬ 
tary  space.” 

The  other  five  battlelabs  are  the  Air 
Expeditionary  Force  Battlelab  at 
Mountain  Home  AFB,  Idaho;  Battle 
Management  Battlelab  at  Hurlburt 
Field,  Fla.;  Unmanned  Air  Vehicle  Bat¬ 
tlelab  at  Eglin  AFB,  Fla.;  Force  Protec¬ 
tion  Battlelab  at  Lackland  AFB,  Texas; 
and  the  Information  Warfare  Battlelab 
at  Kelly  AFB,  Texas.  All  six  battlelabs 
were  operational  by  July  1,  1997. 

Editor’s  Note:  Ozmun  is  with  the  50th 
Space  Wing  Public  Affairs  Office,  Fal¬ 
con  AFB,  Colo. 
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MODELING  AND  SIMULATION 


Modeling  And  Simulation  -  A  New  Role 
for  the  Operational  Tester 

Every  Ideal  Test  is  Tempered  with  Constraints 

STEVEN  K.  WHITEHEAD 


* 


System  Maturity  - ► 


Fleet  Utilization/Release  Recommendation 


he  traditional  role  of  the  inde¬ 
pendent  operational  tester  has  i  ^ 
been  as  the  fleet  users’  repre¬ 
sentative  in  the  acquisition 
process.  It  is  the  operational 
tester’s  responsibility  to  independently 
determine  the  operational  effective¬ 
ness  and  operational  suitability  of  a 
new,  improved,  or  upgraded  system  * 
prior  to  introduction  to  the  fleet.  This  £ 
determination  is  achieved  by  testing  a  g 
production  representative  system,  in 
the  operational  environment,  against 
the  expected  threat,  and  using  fleet 
representative  operators  and  maintain- 
ers.  That  has  been  the  mission  of 
Commander,  Operational  Test  and 
Evaluation  Force  (COMOPTEVFOR) 
for  over  50  years. 

Levels  of  OT&E 

There  are  many  different  levels  of 
operational  test  and  evaluation 
(OT&E)  conducted  by  COMOPTEV¬ 
FOR,  including  developmental  assist 
(DT  Assist),  early  operational  assess¬ 
ment  (EOA),  operational  assessment 
(OA),  initial  operational  test  (lOT), 
software  qualification  testing  (SQT), 
operational  evaluation  (OPEVAL),  veri¬ 
fication  of  correction  of  deficiency 
(VCD),  and  follow-on  operational  test¬ 
ing  and  evaluation  (FOT&E),  all  of 
which,  with  the  exception  of  DT 
Assist,  will  result  in  a  recommendation 
from  COMOPTEVFOR  on  fleet  utiliza¬ 
tion  or  continued  development.  Each 
of  these  levels  of  operational  testing 
(OT)  involve  varying  levels  of  opera¬ 
tional  realism/fidelity,  and  therefore 
will  result  in  varying  levels  of  conclu¬ 


sions  with  regard  to  operational  effec¬ 
tiveness  and  suitability  as  well  as  a 
fleet  release  recommendation.  The 
simple  rule  of  thumb  is:  The  level  of 
confidence  in  projected  system  perfor¬ 
mance  during  actual  fleet  operations  is 
directly  proportional  to  the  fidelity  of 
the  scenario  in  which  the  test  is  con¬ 
ducted  with  regard  to  the  operating 
environment,  including  both  the  phys¬ 
ical  environment  and  system  maturity. 
The  chart  graphically  depicts  this  rule 
of  thumb. 

There  are  two  fundamental  considera¬ 
tions  for  the  operational  tester  that 
apply  to  both  real-world  OT  and  mod¬ 
eling  and  simulation. 


Fidelity  to  operational  environment. 
How  representative  to  the  operational 
environment  is  the  scenario  under 
which  the  data  are  collected?  Given 
the  constraints  placed  upon  even  real- 
world  OT,  actual  test  scenarios  are 
only  “representative”  of  how  the  sys¬ 
tem  will  be  employed.  The  level  of 
fidelity  of  a  model  or  simulation  can 
be  compared  to  the  level  of  fidelity  of 
any  real-world  operational  test.  In  real- 
world  OT,  it  is  not  possible  to  conduct 
a  test  in  actual  combat  conditions; 
therefore,  some  level  of  replication  of 
actual  combat  is  planned  with  as 
many  of  the  variables  and  limitations 
identified  as  possible.  This  process  is 
accepted  because  we  test  to  an  accept- 
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able  level  of  confidence,  with  the 
understanding  that  every  ideal  test  is 
tempered  with  constraints  such  as 
funding,  resource  availability,  technol¬ 
ogy,  etc.  We  continually  leverage  all  data 
sources  to  ensure  the  maximum  use  of 
available  resources.  All  of  this  brings  us 
to  an  operational  test  that  is  less  than 
the  ideal,  and  this  is  accepted  and  ratio¬ 
nal  Since  OT  is  representative  of  fleet 
operations,  there  are  always  tradeoffs 
and  resulting  limitations  to  the  scope  of 
testing.  It  is  anticipated  that  modeling . 
and  simulation  will  be  an  effective  tool  in 
examining  those  areas  that  have,  in  the 
past,  constrained  OT&E. 

System  maturity.  Where  in  the  devel¬ 
opment/procurement  cycle  is  the  sys¬ 
tem?  Early  on  in  the  acquisition/ 
development  cycle,  it  is  not  expected 
that  systems  will  be  able  to  fully  meet 
all  of  their  operational  requirements. 
Systems,  as  well  as  supporting  model¬ 
ing  and  simulation ,  are  expected  to 
mature  over  time,  in  parallel,  with  each 
successive  operational  test  building  upon 
the  information  collected  previously. 

It  is  anticipated  that  models  and  simu¬ 
lations  used  for  system  design  will 
evolve  and  support  those  for  initial 
testing,  and  so  on.  COMOPTEVFOR, 
working  in  parallel  with  the  system 
developers  and  modeling  and  simula¬ 
tion  proponents,  will  gain  additional 
insight  into  how  the  proposed  system 
is  planned  to  meet  its  operational 
requirements. 

Rational  Interpretation 
and  Implementation 

There  is  no  argument  that  modeling 
and  simulation  has  the  potential  to  be  a 
highly  effective  and  efficient  tool  in 
support  of  the  entire  DoD  acquisition 
process  and  especially  OT&E.  It  is  the 
rational  implementation  of  that  tool 
which  is  required.  The  specific  limiting 
uses  of  modeling  and  simulation  are 
delineated  in  DoDD  5000. 2-R,  and 
their  use  is  recommended  for  all  Major 
Defense  Acquisition  Programs 
(MDAP)  and  Major  Automated  Infor¬ 
mation  Systems  (MAIS)  programs. 
Common  sense  dictates  that  this 
approach  also  applies  to  other  than 


MDAPs  and  MAIS;  however,  it  is  the 
interpretation  and  implementation  of 
this  directive  where  common  sense 
plays  the  biggest  role.  The  extent  to 
which  modeling  and  simulation  can  be 
used  to  supplement  OT  is  generally  a 
negotiation  between  the  model  propo¬ 
nent  and  the  operational  tester,  and 
this  is  where  the  new  role  for  the  oper¬ 
ational  tester  is  created. 

In  the  traditional  role,  the  operational 
tester  did  not  set  requirements  or 
thresholds  for  the  system  to  be  tested 
and  evaluated,  and  this  remains  the 
role  for  systems  under  test.  In  the  case 
of  modeling  and  simulation ,  where  users 
of  the  model/simulator  are  the  opera¬ 
tional  testers,  it  is  they  who  must  aid 
in  the  definition  of  the  performance 
output  requirements  of  the  model/ 
simulation.  It  is  the  operational  testers 
who  must  be  satisfied  with  the  level  of 
validation  and  fidelity,  as  the  users,  to 
recommend  accreditation  of  the 
model/simulation  based  on  that  level 
of  satisfaction. 

The  directives  and  instructions  recom¬ 
mending  consideration  of  the  use  of 
modeling  and  simulation  do  not  pre¬ 
scribe  specifically  where  modeling  and 
simulation  should  be  employed.  They 
do,  however,  specifically  state  that 
modeling  and  simulation  cannot  be 
used  exclusively  to  support  beyond 
low  rate  initial  production  decisions. 
Directives  and  instructions  also  do  not 
specifically  prescribe  any  limiting 
amount  of  developmental  data  that  can 
be  used  to  supplement  OT.  The  deci¬ 
sion  as  to  the  amount  of  “other”  data 
(i.e.,  data  not  directly  collected  from  an 
independent  operational  test)  that  are 
used  to  evaluate  a  system  by  the  opera¬ 
tional  tester  is  the  decision  of  the  oper¬ 
ational  tester,  and  this  includes  the 
amount  of  modeling  and  simulation 
used  to  supplement  operational  data. 

Use  of  Modeling  and  Simulation 
inT&E  andOT 

In  support  of  the  Navy  and  DoD 
Vision  for  the  use  of  modeling  and  sim¬ 
ulation  in  T&E,  COMOPTEVFOR  will 
continue  to  work  to  implement  the 
advancements  and  improvements  of 
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the  T&E  process  by  applying  modeling 
and  simulation  technology  to  — 

•  improve  product  quality  and  func¬ 
tionality; 

•  reduce  technical  risk  and  program 
cost; 

•  enhance  performance  assessments; 
and 

•  make  comprehensive  T&E  more 
affordable. 

To  accomplish  this,  COMOPTEVFOR 
will  endeavor  to  make  significant  contri¬ 
butions  to  acquisition  streamlining  by  — 

•  providing  test  environments  that 
can  reduce  acquisition  life-cycle 
costs  and  time  with  no  increase  in 
acceptable  risk;  and 

•  enabling  the  developmental  and 
operational  testers  to  participate  in 
the  model-test-model  process  and 
integrated  product  team  without 
compromising  the  operational 
testers  independence. 

Specifically,  one  method  of  accom¬ 
plishing  this  is  by  leveraging  off  of  the 
extensive  technical  capabilities/knowl¬ 
edge  within  program  offices  to  assist 
in  OT.  The  use  of  program  office 
resources  in  the  understanding  of  sys¬ 
tem  design  and  implementation  of 
operational  requirements  will  in  no 
way  compromise  the  independence  of 
the  operational  tester. 

COMOPTEVFOR  has,  over  the  past 
year,  been  highly  active  in  exploring 
more  efficient  ways  in  which  to  use 
modeling  and  simulation  to  supple¬ 
ment  OT.  The  majority  of  the  endeav¬ 
ors  to  date  have  been  in  accrediting 
hardware/human-in-the-loop  laborato¬ 
ries  and  engineering  facilities.  Accredi¬ 
tation  by  COMOPTEVFOR  is  applica¬ 
tion  and  use-specific.  In  general, 
verification  and  validation  (V&V)  data 
will  be  reusable  to  support  accredita¬ 
tion  decisions  for  other  uses  of  a 
model  or  simulation.  However,  V&V 
data  are  also  gathered  against  specific 


rather  than  general  requirements,  and 
may  need  to  be  amplified  for  a  particu¬ 
lar  application.  The  information  need¬ 
ed  for  accreditation,  and  the  underly¬ 
ing  V&V  processes  and  procedures, 
will  vary  depending  upon  the  nature 
and  scope  of  the  simulation.  In  partic¬ 
ular,  verification,  validation,  and 
accreditation  (W&A)  of  federations 
and  their  associated  federates  is  a  chal¬ 
lenge  that  still  needs  to  be  addressed. 
The  W&A  agents  must  begin  early  in 
the  development  process  to  identify 
the  W&A  requirements  for  federation 
models. 

Involve  Operational  Testers  Early 

As  Navy  operational  testers  are  not 
software  or  systems  engineers  but 
rather  are  operators  with  widely  vary¬ 
ing  degrees  of  technical  education,  it  is 
imperative  that  the  operational  testers 
be  involved  early  and  are  sufficiently 
educated  to  understand  the  basic  prin¬ 
ciples  and  uses  of  modeling  and  simula¬ 
tion.  To  this  end,  it  was  necessary  for 
COMOPTEVFOR  to  develop  a  list  of 
fundamental  questions  for  the  opera¬ 
tional  test  director  (OTD).  The 
answers  to  these  questions  will  assist 
the  OTD  in  establishing  a  baseline 
knowledge  level  with  regard  to  each 
modeling/simulation  development 
and  utilization. 

Q 

What  is  the  reason  for  the  initial  develop¬ 
ment  of  the  model,  and  what  is  its  simi¬ 
larity  to  the  current  application?  Is  there 
a  requirements  document  for  the  model 
and  a  software  design  specification  for  the 
initial  implementation  and  for  any  modi¬ 
fications? 

Q 

What  is  the  developer's  reputation,  Soft¬ 
ware  Engineering  Institute  rating,  and 
model  development  experience?  Can  the 
developer  provide  metrics  on  software 
maturity,  complexity. ;  requirements  trace- 
ability ;  design  stability,  and  depth  and 
breadth  of  testing? 

Q 

What  are  the  hardware,  software,  person¬ 
nel,  data,  and  security  requirements  asso¬ 
ciated  with  using  the  model?  What  is  the 


schedule  for  model  development  and 
model  V&V  activities? 

Q 

What  is  the  configuration  management 
(CM)  status  of  the  model  and  its  associ¬ 
ated  databases?  Does  the  CM  process 
have  these  four  characteristics:  ( 1 )  a  well- 
defined  baseline;  (2)  standard  baseline 
test  cases  and  data  sets;  (3)  well-defined, 
coordinated,  and  supported  testing  pro¬ 
gram;  and  (4)  current ,  thorough  docu¬ 
mentation? 

Q 

What  V&V  has  been  accomplished,  or  is 
planned,  to  establish  model  credibility? 

Q 

What  modeling  and  simulation  docu¬ 
mentation  is  available  (types  of  documen¬ 
tation,  detail,  accuracy ;  and  currency )? 

Q 

What  are  the  known  limitations  or  prob¬ 
lems  with  the  model?  (A  good  configura¬ 
tion  management  system  has  such  a  list 
readily  available.) 

Operational  Testers  do  not  “test”  or 
verify  models  or  simulations.  They  are, 
however,  closely  involved  in  the  valida¬ 
tion  process.  The  Draft  COMOPTEV- 
FORINST  5000.X  establishes  proce¬ 
dures  on  the  use  of  models  to  support 
OT&E  and  describes  the  information 
necessary  for  accreditation  by 
COMOPTEVFOR.  It  is  the  model  pro¬ 
ponent’s  responsibility,  in  conjunction 
with  COMOPTEVFOR,  to  - 

•  develop  plans  to  use  modeling  and 
simulation  in  OT,  which  includes  a 
description  of  the  system,  test  objec¬ 
tives,  modeling  and  simulation  objec¬ 
tives,  and  a  test  schedule; 

•  develop  V&V  to  support  accredita¬ 
tion  for  the  application;  and 

•  provide  a  V&V  plan,  V&V  reports, 
and  other  support  documentation, 
such  as  model  user  guides,  analyst 
notebooks,  configuration  manage¬ 
ment  plans,  software  development 
policy  and  procedures,  and  software 
process  review  reports. 


The  accreditation  package  contains  at 
least  the  minimum  documentation 
required  by  DoD  5000.59P  and  Draft 
SECNAVINST  5200.XX. 

Conclusions 

As  a  tool  to  supplement  for  limited 
assets,  it  is  COMOPTEVFOR  policy 
that  the  modeling  and  simulation  will 
not  replace  actual  operational  assets. 
Modeling  and  simulation  is  a  tool  to 
more  effectively  and  efficiendy  employ 
the  limited  assets  available.  Modeling 
and  simulation  should  not  be  used  to 
extrapolate  system  performance.  The 
Navy’s  Draft  Test  and  Evaluation  Mod¬ 
eling  and  Simulation  Master  Plan 
includes  the  documentation  require¬ 
ments,  with  formats,  for  the  use  of 
modeling  and  simulation  in  OT.  The 
accreditation  plan  format,  accredita¬ 
tion  report  format,  and  verification 
and  validation  report  format  are  sug¬ 
gested  formats  and  can  be  tailored  to 
each  application. 

While  OT  must  remain  “operational,” 
modeling  and  simulation  can  be  used 
very  successfully  in  test  planning, 
rehearsals,  training,  post-test  analysis, 
and  in  limited  cases,  the  test  itself. 
Specific  guidance  on  when  modeling 
and  simulation  can  be  successfully 
applied  cannot  be  a  cookbook 
approach.  Each  program  must  exam¬ 
ine  the  testing  areas  that  could  be 
more  effectively  executed  using  model¬ 
ing  and  simulation .  In  some  cases,  the 
use  of  modeling  and  simulation  may  be 
more  expensive  than  traditional  test¬ 
ing,  but  yield  results  that  would  be 
impossible  to  obtain  using  traditional 
testing.  In  all  cases,  the  decision  mak¬ 
ers  and  the  operational  testers  must 
assess  the  value  added  by  modeling  and 
simulation  and  determine  the  most 
cost-effective  testing  plan. 

Operational  testers  must  continue  to 
participate  in  the  modeling  and  simula¬ 
tion  initiative  that  will  form  the  basis 
for  future  use  of  emerging  technolo¬ 
gies  to  ensure  OT&E  specific  issues 
are  incorporated.  Additionally,  an 
aggressive  effort  must  be  made  to 
identify  and  use  the  full  capability  of 
modeling  and  simulation  within  OT&E. 
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MODELING  AND  SIMULATION 


SPY-1  D(V)  Models  and  Simulations 
Support  Operational  Testing  in  a 
Remote  New  Jersey  Cornfield 

PEO,  Developer,  Operational  Tester 
Combination  Works  Smarter,  Placing 
Best  Technology  in  Warfighters'  Hands 

LT.  CMDR.  HARRY  M.  CROYDER,  U.S.  NAVY 
CMDR.  WILLIAM  P.  ERVIN,  U.S.  NAVY  •  DR.  DAVID  S.  MAZEL 


ccredited  models  and  simula¬ 
tions  make  land-based  testing 
of  the  SPY-1  radar  family  more 
credible  than  ever  before.  This 
article  is  about  one  such  oper¬ 
ational  radar  test,  conducted  in  a 
remote  New  Jersey  cornfield. 

Also  in  this  article,  we  explain  the  veri¬ 
fication,  validation,  and  accreditation 
of  the  SPY-ID(V)  program  models  and 
simulations,  and  how  this  process  not 
only  ensures  the  proper  use  of  high- 
fidelity,  thoroughly  understood  models 
and  simulations,  but  also  enhances 
the  realism  and  credibility  of  opera¬ 
tional  testing.  Further,  we  describe 
development  and  application  of  this 
accreditation  process  in  support  of  the 
recent  SPY-ID(V)  radar  test;  focus  on 
the  managerial  versus  the  technical 
aspect  of  this  process;  and  present 
potentially  useful  ideas  to  organiza¬ 
tions  involved  with  modeling  and  sim¬ 
ulation  in  the  operational  test  and 
evaluation  arena. 

Navy's  SPY-1  D(V) 

Strategy  Decision 

In  1994,  the  Navy  faced  an  important 
acquisition  strategy  decision  -  impor¬ 
tant  because  the  AEGIS  SPY  radar  sys¬ 
tem  is  completely  integrated  into  the 

Croyder  is  a  surface  warfare  officer  with  over  79  years  of  service.  He  is  currently  assigned  to  Commander,  Operational  Test  and  Evaluation  Force  (COMOPTEVFOR), 
Norfolk,  Va.,  as  the  operational  test  director  for  all  AEGIS  programs.  Ervin  is  a  surface  warfare  officer,  currently  assigned  as  section  head  for  surface-to-air  missiles 
at  COMOPTEVFOR.  Maze!  is  a  research  analyst  for  The  CNA  Corporation,  Center  for  Naval  Analyses.  He  holds  a  PhD.  and  is  currently  on  temporary  assignment 
to  COMOPTEVFOR. 


;  AEGIS  Combat  System  Engineering  Devel- 
y  opment  Site  (CSEDS),  Home  of  the  “Corn- 
:  .  field  Cruiser" 

Photo  courtesy  Unisys  Corporation 

AEGIS  ship,  and  it  takes  five  years  to 
build  a  ship.  Two  options  emerged  for 
consideration: 


Option  1.  Produce  and  install  a 
single  SPY-ID(V)  radar  in  a 
new  construction  DDG  51-class 
ship. 

Option  2.  Use  the  land-based 
test  site  to  test  operationally  the 
engineering  development 
model  of  the  SPY-ID(V)  radar. 


86  PM  :  SEPTEMBER-OCTOBER  1997 


Option  1  would  cause  the  interrup¬ 
tion  of  SPY-ID  radar  production  and 
create  a  unique  operational  ship  for 
the  sole  purpose  of  at-sea  testing  to 
support  a  low-rate  initial  production 
acquisition  decision.  This  option 
would  have  the  advantage  of  testing  in 
the  operational  environment,  but  the 
disadvantage  of  delaying  fleet  intro¬ 
duction  of  SPY-ID(V)  radars  for  up  to 
five  years  and  incurring  additional 
costs  for  creating  a  unique  asset  and 
conducting  two  SPY-ID(V)  produc¬ 
tion  starts  versus  one. 

Option  2  called  for  land-based  testing 
to  support  a  low-rate  initial  production 
acquisition  decision  without  interfer¬ 
ing  with  current  radar/ship  produc¬ 


tion.  This  option  had  the  advantage  of 
making  the  acquisition  decision  in 
1996  vice  2003-plus,  but  the  disadvan¬ 
tage  of  testing  in  a  land-based  operat¬ 
ing  environment. 

Key  to  the  Navy’s  SPY-ID(V)  strategy 
decision  was  a  determination  that 
land-based  testing  was  adequate  to 
support  a  low-rate  initial  production 
decision.  Toward  that  end,  the  Navy 
planned  to  conduct  this  land-based 
testing  at  its  Combat  Systems  Engi¬ 
neering  Development  Site  (CSEDS)  in 
Moorestown,  New  Jersey.  Due  to  its 
land-locked  location,  CSEDS’  charac¬ 
teristics  are  vastly  different  from  any 
shipboard  environment,  and  those  dif¬ 
ferences  remained  to  be  assessed. 

The  CSEDS  facility  is  50  miles  from 
the  Atlantic  Ocean  in  a  location  that 
prohibits  low-flying  aircraft  and 
severely  restricts  chaff  and  electronic 
jamming  activities.  Any  test  scenarios 
involving  fixed  wing  aircraft,  heli¬ 
copters,  chaff,  and  jamming  must  be 


C  SPY-1  D(V)  Radar.  The  AEGIS  SPY  radar  system  is 
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conducted  in  areas  that  do  not  inter¬ 
fere  with  commercial  airways,  nearby 
subdivisions,  or  local  farm  animals. 
Site  characteristics  bear  little  resem¬ 
blance  to  the  at-sea  operating  environ¬ 
ment  of  dynamic  sea  clutter,  multipath 
low  elevation  propagation,  and  pitch¬ 
ing  and  yawing  conditions  a  radar  will 
operate  in  when  installed  in  a  Navy 
ship.  The  testing  methods  for  SPY- 
lD(V)’s  new  capabilities  were  all 
adversely  impacted  by  CSEDS’  site 
limitations. 

To  help  make  the  test  adequacy  deter¬ 
mination,  the  Assistant  Secretary  of 
the  Navy  (Research,  Development, 
and  Acquisition)  (ASN[RDA])  commis¬ 
sioned  an  independent  advisory  com¬ 
mittee  to  investigate  the  SPY-lD(V)’s 
capabilities  and  CSEDS  characteristics. 
After  assessing  risk  mitigation,  techni¬ 
cal  risks,  and  test  adequacy,  this  inde¬ 
pendent  committee  concluded  that, 
with  the  use  of  models  and  simula¬ 
tions,  the  radar  could  be  tested  well 
enough  to  support  the  low-rate  initial 
production  decision.  Based  in  part  on 
this  conclusion,  ASN(RDA)  chose 
Option  2  and  signed  an  Acquisition 
Decision  Memorandum  authorizing 
land-based  operational  testing  at 
CSEDS. 

ASN(RDA)’s  deci¬ 
sion  complement¬ 
ed  the  growing 
trend  within  the 
Department  of  De¬ 
fense  (DoD)  to 
find  alternatives 
for  the  ever-increas¬ 
ing  costs  and  rap- 
idly  shrinking  re¬ 
sources  associated 
with  test  and  evalu- 
ation  requirements, 
particularly  require¬ 
ments  associated 
v  with  field  tests. 

...  One  alternative  is 

f  the  use  of  models 

toward  models  and 
8§ j£j  simulations  as  a 

waY  t0  cut  expens- 
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es  in  developmental  and  operational 
testing.  Real-world  assets  such  as  very 
small  targets,  aircraft  services,  and 
missile  firings  are  becoming  increas¬ 
ingly  scarce  and  expensive.  Some 
acquisition  programs  have  been  using 
models  and  simulations  for  years  and 
have  established  methodologies  for 
conducting  verification  and  valida¬ 
tion. 

The  Verification,  Validation,  and 
Accreditation  Process 

To  the  Navy’s  independent  test  agency 

-  Commander,  Operational  Test  and 
Evaluation  Force  (COMOPTEVFOR) 

-  the  idea  of  using  models  and  simu¬ 
lations  instead  of  actual  field  opera¬ 
tions  to  validate  at-sea  systems’ 
performance  was  a  departure  from  tra¬ 
ditionally  accepted  testing  methodolo¬ 
gy.  To  the  COMOPTEVFOR  staff,  who 
experienced  and  well  understood  at- 
sea  realities,  the  modeling  of  the  SPY- 
lD(V)’s  new  capabilities  for  opera¬ 
tional  applications  had  little  credibility 
because  CSEDS  is  land-locked. 

COMOPTEVFOR  supported  the  move 
toward  models  and  simulations  by 
developing  a  command  concept  and 
procedure  that  outlined  how  models 
and  simulations  fits  into  operational 
testing.  Involving  a  process  called  veri¬ 
fication,  validation,  and  accreditation, 
this  concept  calls  for  a  program  execu¬ 
tive  office  to  verify  and  validate  all  the 
models  and  simulations  it  requires  to 
perform  necessary  developmental  and 
engineering  tests.  Ideally,  the  verifica¬ 
tion  and  validation  process  should  sat¬ 
isfy  the  program  executive  office  that 
the  selected  models  and  simulations 
function  as  expected.  When  the  pro¬ 
gram  executive  office  is  satisfied,  it  for¬ 
mally  accepts  the  models  and  simula¬ 
tions  for  use  in  developmental  testing. 
This  formal  acceptance  is  called  certifi¬ 
cation,  and  is  the  measure  of  the  pro¬ 
gram  office’s  confidence  in  its  model. 
After  certification,  the  program  execu¬ 
tive  office  directs  the  model’s  use  in 
the  developmental  test  strategy.  If  the 
models  and  simulations  will  be  used 
in  an  operational  test,  COMOPTEV¬ 
FOR  must  accredit  the  models  and 
simulations  for  a  specific  purpose 


within  that  test.  Accreditation  is  the 
COMOPTEVFOR  formal  acceptance 
of  the  validated  models  and  simula¬ 
tions.  COMOPTEVFOR  always  consid¬ 
ers  certification  a  prerequisite  to 
accreditation. 

Step  1.  The  Simulation  Management 
Plan  (SMP).  Neither  the  Program 
Executive  Office  Surface  Combatants- 
AEGIS  Program  (PEO  SC-AP)  nor 
COMOPTEVFOR  possessed  the  expe¬ 
rience  or  the  infrastructure  to  support 
any  of  the  new  models  and  simula¬ 
tions  initiatives,  including  verification, 
validation,  and  accreditation.  Some  of 
the  basic  concepts  were  there  such  as 
certification  and  accreditation,  but  few 
of  the  real-world  mechanics.  Those 
mechanics  had  to  be  created. 

As  the  first  step,  we  found  a  working 
models  and  simulations  organization. 
As  a  result  of  using  models  and  simu¬ 
lations  for  years,  the  Tomahawk  Cruise 
Missile  Program  possessed  practical 
experience,  which  it  willingly  shared. 
The  PEO  SC-AP  and  COMOPTEVFOR 
staff  members,  however,  faced  the 
daunting  task  of  mastering  the  Toma¬ 
hawk  methodology;  the  COMOPTEV¬ 
FOR  verification,  validation,  and 
accreditation  instruction;  the  program 
executive  office  and  COMOPTEVFOR 
goals;  and  the  time  and  financial  con¬ 
straints  on  the  entire  process.  Once 
they  digested  all  these  elements, 
the  program  executive  office  and 
COMOPTEVFOR  staffs  jointly  authored 
a  verification,  validation,  and  accredita¬ 
tion  plan,  called  the  SPY-ID(V)  Radar 
System  DT/OT  Simulation  Manage¬ 
ment  Plan  (SMP). 

First  SMP  Component  -  The  Goals 
The  establishment  of  goals  by  each 
participating  office  is  the  first  compo¬ 
nent  of  the  SMP.  Once  established, 
each  office  must  clearly  understand 
the  goals  of  all  other  offices  and  joint¬ 
ly  design  a  framework  that  will  mutu¬ 
ally  support  the  achievement  of  all 
goals. 

Accreditation  of  those  models  that 
supported  its  mission  —  the  opera¬ 
tional  test  -  was  COMOPTEVFOR’s 


primary  goal.  In  this  case,  accredita¬ 
tion  required  seven  models/simula¬ 
tions/simulators/stimulations.  Only 
after  a  thorough  review  of  the  verifica¬ 
tion  and  validation  process  to  deter¬ 
mine  the  fidelity  of  each  model  in  sup¬ 
porting  operational  testing,  was 
accreditation  awarded.  Prior  to  accred¬ 
itation,  we  prepared  and  reviewed  the 
following  required  documents  for 
each  model  (discussed  at  length  in 
subsequent  paragraphs): 

•  Simulation  Validation  Plan 

•  Simulation  Validation  Report 

•  Simulation  Version  Description 
Document 

•  Program  Executive  Office  Certi¬ 
fication 

No  requirement  exists  that  any  model 
must  exacdy  replicate  the  real  world; 
in  other  words,  no  model  is  expected 
to  be  a  “perfect”  empirical  representa¬ 
tion. 

Alternately,  one  of  the  program  execu¬ 
tive  office’s  major  goals  was  the 
accreditation  of  its  models  and  simula¬ 
tions.  Accreditation  meant  that  the 
SPY-ID(V)  models  and  simulations 
were  credible  enough  to  conduct  the 
test  strategy  outlined  in  ASN(RDA)’s 
Acquisition  Decision  Memorandum. 
Accreditation  also  meant  that  an  out¬ 
side  activity  reinforced  the  program 
executive  office’s  reputation  for 
enforcing  standards.  Since  certifica¬ 
tion  was  a  prerequisite  to  accredita¬ 
tion,  the  SMP  outlined  the  program 
executive  office’s  certification  require¬ 
ments  as  well. 

Second  SMP  Component  — 

Verification  and  Validation  Method 
The  other  major  component  in  the 
SMP  is  the  actual  verification  and  vali¬ 
dation  execution  framework.  The  pre¬ 
ferred,  overarching  theoretical  concept 
of  verification  and  validation  calls  for  a 
disinterested  third  party  to  accomplish 
validation.  This  type  of  validation  is 
known  as  independent  verification 
and  validation.  For  the  SPY-ID(V),  nei- 
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ther  the  time  nor  the  money  existed  to 
contract  such  a  party  to  independent 
verification  and  validation  —  all  seven 
required  models  and  simulations. 
Instead,  the  SMP  authorized  an  inter¬ 
nal  verification  and  validation  method, 
the  use  of  which  represented  a  need  to 
mitigate  any  credibility  risk  to  the  pro¬ 
gram.  This  meant  that  the  models  and 
simulations  developers  would  validate 
their  own  models  with  program  execu¬ 
tive  office  and  COMOPTEVFOR  over¬ 
sight  instead  of  independent  verifica¬ 
tion  and  validation. 

Again,  in  the  interest  of  time  and 
money,  the  SMP  did  not  require  new 
data  collection.  In  other  words,  for  cer¬ 
tain  models  the  developers  were  not 
tasked  to  acquire  new  empirical  data 
to  support  verification  and  validation. 
New  collection  and  analysis  of  atmos¬ 
pheric  propagation,  sea  clutter,  or  live 
missile  telemetry  data  was  impractical. 
This  information  already  existed  in 
several  places  and  could  be  used  at 
significant  time  and  cost  savings. 

Third  SMP  Component  -  Credibility 
Next,  PEO  SC-AP  and  COMOPTEV¬ 
FOR  agreed  that  their  staffs  must 
maintain  ruthless  self-discipline  to 
reduce  risk  and  ensure  credibility 
since  independent  verification  and  val¬ 
idation  would  not  be  used.  All  verifica¬ 
tion,  validation,  and  accreditation  pro¬ 
cedures,  results,  and  discussions 
would  be  open  to  outside  agencies’ 
inspection.  This  openness  philosophy 
was  the  cornerstone  of  the  entire 
effort’s  success. 

Fourth  SMP  Component  - 
The  Framework 

Finally,  the  SMP  provided  the  organi¬ 
zational  structure  to  achieve  the  goals 
and  execute  the  verification  and  vali¬ 
dation  method.  This  structure  consist¬ 
ed  of  the  Simulation  Management 
Board  (SMB)  and  the  Simulation  Con¬ 
trol  Panel  (SCP).  The  SMP  required 
the  use  of  the  SMB  and  the  SCP  and 
provided  an  executive  summary  of 
their  functions.  The  SMP  also 
described  each  one’s  membership  and 
its  role  in  accomplishing  certification 
and  accreditation. 


SPY-ID(V)  Accreditation  Process 


Determine  Mutual  Goals 


Author  Joint  SMP 


1.  List  Certification  Requirements 

2.  List  Accreditation  Requirements 

3.  Outline  W&A  Process 

1 .  Determine  V&V  Approach  on  a  Case  Basis 

2.  Approve  Developers’ SVPs 

3.  Oversee  V&V  Process 

4.  Examine  V&V  Results:  Fidelity,  Capabilities,  &  Limitations 


Return  to  Developer 


1.  Examine  SCP  Results 

2.  Determine  Rigor  in  Process 


Return  to  SCP 


Certifying  Authority 


^ - ►  Return  to  SMB 

Y-^ - 1 


Developmental  Tests 


Accreditation  Authority 


Return  to  SMB 


Operational  Test 


Simulation  Management  Plan 
Verification,  Validation,  and  Accreditation 

Simulation  Control  Panel 
Simulation  Validation  Report 
Simulation  Version  Description  Document 
Simulation  Management  Board 
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Step  2.  The  Simulation  Control 
Panel  (SCP).  The  SCP  provided  the 
working  technical  oversight  of  the  veri¬ 
fication  and  validation  process.  Its 
composition  included  mainly  techni¬ 
cal  personnel,  who  well  understood 
their  respective  models  and  simula¬ 
tions,  as  well  as  AEGIS  combat  system 
technical  representatives.  Part  of  the 
SCP  s  function  was  to  promote  a  tech¬ 
nical  exchange. 


MM 

When  the  SCP 
leadership  believed 
they  achieved 


The  SCP  -  Its  Membership 
The  SCP’s  chairperson  was  the  SPY- 
1D(V)  program  manager’s  assis¬ 
tant.  The  co-chairperson  was  the 
COMOPTEVFOR  operational  test 
director  for  the  SPY  radar  program. 
These  two  individuals  directed  the 
oversight  process.  It  is  important  to 
note  that  both  co-chairpersons  had  to 
be  in  agreement  for  any  item  to  pass 
the  SCR  Other  members  included 
technical  representatives  from  the 
three  companies  who  developed  the 
models  and  simulations,  namely 
Lockheed  Martin  (Government  Elec¬ 
tronic  Systems)  Corporation,  Technol¬ 
ogy  Service  Corporation,  and  Systems 
Engineering  Group.  Additionally,  the 
Naval  Surface  Warfare  Center  and 
AEGIS  Technical  Representative  pro¬ 
vided  technical  support  to  the  program 
executive  office  chairperson,  and  the 
Center  for  Naval  Analyses  supported 
the  COMOPTEVFOR  co-chairperson. 

The  SCP  -  Its  Function 
As  previously  mentioned,  the  SCP’s 
charter  was  to  perform  the  working- 
level  oversight  of  the  verification  and 
validation  process.  Toward  that  end, 
the  membership  devoted  a  good  deal 
of  time  and  effort  to  understanding 
and  defining  the  seven  models  and 
simulations.  When  the  SCP  leadership 
believed  they  achieved  a  sufficient 
understanding  of  each  model  and  sim¬ 
ulation,  they  asked  the  developer  to 
propose  a  verification  and  validation 
plan  based  on  its  assets  and  the  data 
available.  When  the  developer  eventu¬ 
ally  submitted  a  proposal,  the  mem¬ 
bership  then  discussed  it  at  length  and 
selected  the  actual  process  the  devel¬ 
oper  would  use  to  validate  the  models 
and  simulations. 


a  sufficient 
Understanding  of 
each  model  and 


Most  of  the  early  meetings  centered 
around  selecting  the  proper  verifica¬ 
tion  and  validation  method.  Some¬ 
times  these  discussions  were  rather 
frank  and  resulted  in  some  strong  dis¬ 
agreements,  but  fortunately  the  SMP 
did  not  require  unanimity.  Once  the 
co-chairpersons  accepted  the  valida¬ 
tion  proposal,  the  developers  proceed¬ 
ed  to  write  the  Simulation  Validation 
Plan.  The  SCP  met  frequently  to  moni¬ 


tor  validation  progress.  Sometimes,  of 
necessity,  the  SCP  changed  verification 
and  validation  procedures  because  the 
developer  found  a  better  way  or  dis¬ 
covered  the  current  method  wasn’t 
working  as  planned.  The  SCP  mem¬ 
bership  carefully  reviewed  validation 
progress  and  early  results  to  ensure 
they  met  the  objectives  initially  out¬ 
lined  in  the  SMP.  As  verification  and 
validation  progressed,  the  developers 
began  to  write  the  Simulation  Valida¬ 
tion  Report  and  the  Simulation  Ver¬ 
sion  Description  Document. 

The  Simulation  Validation  Plan 
Groundwork.  The  SMP  required  a  sep¬ 
arate  Simulation  Validation  Plan  for 
each  model  and  simulation.  As  previ¬ 
ously  noted,  early  SCP  meetings  cen¬ 
tered  around  determining  which  verifi¬ 
cation  and  validation  method  to 
employ  for  each  model  and  simula¬ 
tion.  During  those  determinations  and 
in  order  to  author  the  Simulation  Vali¬ 
dation  Plan,  several  questions  re¬ 
mained  to  be  answered,  or  at  least 
addressed: 

Is  the  model  and  simulation  a  model?  (A 
model  is  defined  as  a  physical,  mathe¬ 
matical,  or  otherwise  logical  represen¬ 
tation  of  a  system  entity,  phenomenon, 
or  process.) 

Is  the  model  and  simulation  a  simula¬ 
tion?  (A  simulation  is  defined  as  a 
method  for  implementing  a  model 
over  time,  or  where  real-world  and 
conceptual  systems  are  reproduced  by 
a  model.) 

For  what  purpose  will  the  model  and 
simulation  be  used? 

What  are  the  capabilities  and  limitations 
of  each  model  and  simulation? 

What  value  will  the  model  and  simula¬ 
tion  add  to  the  operational  test? 

How  will  use  of  each  model  and  simula¬ 
tion  impact  the  operational  testers  abili¬ 
ty  to  formulate  conclusions? 

How  does  the  model  interoperate  with  the 
other  six  models? 
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What  options  exist  within  the 
time/money/ data  constraints  to  verify 
and  validate  each  model? 

In  practice,  the  SCP  answered  some  of 
these  questions  only  after  they 
approved  the  Simulation  Validation 
Plan,  and  the  interoperability  issue  was 
never  completely  addressed.  The  SCP 
intended  the  verification  and  valida¬ 
tion  process  to  be  flexible.  When  the 
panel  found  a  better  way,  they  altered 
the  process  and  sometimes  changed 
an  answer  too.  Once  the  SCP  assem¬ 
bled  sufficient  information,  it  ad¬ 
dressed  requirements  for  the  Simula¬ 
tion  Validation  Plan. 

Two  Simulation  Validation  Plan  Re¬ 
quirements.  The  first  Simulation  Vali¬ 
dation  Plan  requirement  was  the  selec¬ 
tion  of  the  right  method  based  on  the 
SCP’s  understanding  of  the  models 
and  simulations  .  As  a  result,  the  SMP 
mandated  that  the  verification  and  val¬ 
idation  process  use  at  least  one  of 
three  possible  methods: 

•  Model-to-Real-World  Compari¬ 
son 

•  Model-to-Model  Comparison 

•  Code  Analysis 

For  SPY-ID(V),  a  model-to-real-world 
example  was  the  simulation  that  repre¬ 
sented  small  radar  cross-section  tar¬ 
gets.  Because  no  real-world  targets 
existed,  the  developer  used  the  model- 
to-real-world  simulation,  attaching  a 
physical  sphere  to  a  balloon  and 
launching  it  into  the  air.  This  sphere 
had  a  known  cross-section  that  fluctu¬ 
ated  in  the  real  environment.  As  it 
floated  away,  the  SPY-ID(V)  radar 
tracked  the  sphere.  It  also  tracked  a 
target  simulation  constructed  with  the 
same  cross-section.  Unlike  the  sphere, 
however,  the  target  simulation  pos¬ 
sessed  no  cross-section  fluctuating 
capability.  We  then  compared  the 
sphere’s  cross-section,  as  observed  by 
the  radar,  to  the  simulation’s  cross-sec¬ 
tion  as  observed  by  the  radar.  Results 
determined  the  corrective  action  nec¬ 
essary  to  improve  the  simulation. 


A  model-to-model  example  was  the 
sea  clutter  simulation.  We  used  this 
simulation  because  CSEDS  is  a  long 
way  from  the  ocean.  The  simulation 
was  actually  a  composite  of  two  mod¬ 
els  and  simulations  —  a  mathematical 
model,  representing  the  sea  clutter 
phenomenon;  and  a  hardware  genera¬ 
tor,  which  implemented  the  model 
into  the  system  such  that  the  radar 
could  observe  the  sea  clutter.  Valida¬ 
tion  of  the  generator’s  implementation 
ability  compared  the  mathematical 
model  with  the  generator’s  simulation. 
The  results  initiated  a  plan  of  action. 

The  second  requirement  stipulated 
that  the  known  capabilities  and  limita¬ 
tions  of  the  models  and  simulations 
be  stated.  Every  Simulation  Validation 
Plan  included  a  list  of  the  known 
capabilities  and  limitations  of  its 
model  to  preclude  future  misunder¬ 
standings.  The  unforeseen  benefit  of 
this  requirement  was  the  discovery 
that  the  “known”  capabilities  and  limi¬ 
tations  listed  in  the  Simulation  Valida¬ 
tion  Plan  were  not  necessarily  the  same 
ones  revealed  later  during  verification 
and  validation. 

As  verification  and  validation  pro¬ 
gressed,  the  SCP  began  to  author  the 
next  two  required  documents,  the  Sim¬ 
ulation  Validation  Report  and  the  Sim¬ 
ulation  Version  Description  Document. 

The  Simulation  Validation  Report 
The  Simulation  Validation  Report  was 
the  written  report  of  results  achieved 
during  verification  and  validation.  It 
contained  an  executive  summary  and 
a  technical  analysis  section.  Included 
in  the  Simulation  Validation  Report 
were  validation  details  such  as  — 

•  a  description  of  the  actual  vali¬ 
dation  procedure; 

•  a  discussion  of  why  that  proce¬ 
dure  differed  from  the  one  out¬ 
lined  in  the  Simulation  Valida¬ 
tion  Plan;  and 

•  a  list  of  capabilities  and  limita¬ 
tions  confirmed  by  the  verifica¬ 
tion  and  validation.  Where  the 


Simulation  Validation  Plan  and 
Simulation  Validation  Report 
lists  differed,  the  developer 
added  an  explanatory  note. 

The  Simulation  Version  Description 
Document 

The  Simulation  Version  Description 
Document  briefly  described  the  com¬ 
puter  program  configuration  manage¬ 
ment  that  supported  the  models  and 
simulations.  The  developer  met  this 
SMP  requirement  chiefly  through  a 
related,  non-accreditation  event  called 
a  COMOPTEVFOR  Software  Quick- 
look.  A  Software  Quicklook  provided 
COMOPTEVFOR  with  a  basic  under¬ 
standing  of  a  developer’s  software 
management  program. 

The  program  executive  office  had  pre¬ 
viously  encouraged  the  conduct  of  a 
Software  Quicklook  to  promote 
COMOPTEVFOR’s  understanding  of 
configuration  management  issues.  A 
thorough  review  of  the  Quicklook 
confirmed  that  the  prime  developer 
followed  accepted  software  configura¬ 
tion  management  procedures,  further 
increasing  COMOPTEVFOR’s  confi¬ 
dence  in  the  models  and  simulations. 
Since  the  Quicklook  is  not  a  verifica¬ 
tion,  validation,  and  accreditation 
requirement,  it  did  not  eliminate  the 
accreditation  requirement  for  a  Simu¬ 
lation  Version  Description  Document. 
Plowever,  using  Quicklook  data,  the 
SCP  could  streamline  the  document. 

Now  verification  and  validation  was 
complete.  The  SCP  had  written  a  Sim¬ 
ulation  Validation  Plan,  and  the  devel¬ 
opers  had  executed  it.  The  approved 
Simulation  Validation  Report  con¬ 
tained  an  executive  summary  and  the 
technical  results.  The  Simulation  Ver¬ 
sion  Description  Document  was  com¬ 
plete. 

The  co-chairpersons  agreed  to  move 
the  verification,  validation,  and  accred¬ 
itation  process  forward.  The  next  step 
was  to  convene  the  Simulation  Man¬ 
agement  Board. 

Step  3.  The  Simulation  Management 
Board  (SMB).  The  SMB  was  a  four- 
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member  board,  chaired  by  the  SPY- 
ID^)  program  manager.  Its  purpose 
was  to  recommend  certification  to  the 
program  executive  office  certifying 
officer.  Prior  to  recommending  certifi¬ 
cation,  it  evaluated  the  Simulation  Vali¬ 
dation  Reports  provided  by  the  SCP. 
The  SMB  voting  members  were  the 
chairperson,  the  PEO  SC-AP  models 
and  simulation  division  head,  and  an 
AEGIS  Technical  Representative  senior 
staff  member.  The  COMOPTEVFOR 
Assistant  Chief  of  Staff  for  Surface 
Warfare  acted  as  the  single,  nonvoting 
advisory  member. 

The  SMB  acted  to  satisfy  its  member¬ 
ship  that  the  verification  and  valida¬ 
tion  had  been  rigorously  executed.  In 
that  regard,  the  board  consulted  the 
COMOPTEVFOR  advisory  member 
for  the  accreditation  authority’s  per¬ 
spective  on  the  verification  and  val¬ 
idation  results.  When  the  vote  was 
unanimous,  the  board  forwarded  a 
certification  recommendation  to  the 
proper  authority  at  the  program  execu¬ 
tive  office.  When  the  vote  was  not 
unanimous,  the  board  returned  the 
product  to  the  SCP  for  additional 
work. 

The  SMB/SCP  membership  intended 
their  proceedings  to  be  an  open 
process.  Interested  parties  from  the 
Director,  Operational  Test  and  Evalua¬ 
tion  and  the  Institute  for  Defense 
Analyses  had  a  standing  invitation  to 
attend  either  board/panel.  The  mem¬ 
bership  extended  this  standing  invita¬ 
tion  for  two  purposes: 

•Without  specific  DoD  guid¬ 
ance,  the  SPY-1D(V)  joint 
verification,  validation,  and 
accreditation  effort  was  some¬ 
what  “experimental.”  Agencies 
closer  to  DoD  might  be  able  to 
provide  additional  perspectives 
on  the  future  evolution  of 
models  and  simulations 
policy 

•  The  demonstration  of  the  rigor¬ 
ous,  disciplined  process  should 
be  witnessed  and  not  merely 
advertised. 


Step  4.  Certification  and  Accredita¬ 
tion  The  SMB  chairman  briefed  the 
certifying  authority  on  the  results  and 
recommendations  of  the  SMB.  This 
authority  certified  the  recommended 
models  and  simulations  when  con¬ 
vinced  that  the  SMB  had  applied  the 
requisite  tough  examination  required 
by  the  SMP  tenet  of  self-discipline. 
After  the  program  executive  office 
completed  its  internal  administration, 
the  certifying  official  then  sent  an  offi¬ 
cial  letter  of  certification  to  the  accredi¬ 
tation  authority. 

Upon  receipt,  the  OPTEVFOR  opera¬ 
tional  test  director  briefed  the  accredit¬ 
ing  officer  on  the  certification  letter. 
Included  in  the  brief  was  a  synopsis  of 
the  technical  details  from  each  Simula¬ 
tion  Validation  Report,  including  capa¬ 
bilities  and  limitations;  the  intended 
use  of  the  models  and  simulations  in 
the  operational  test;  and  an  assess¬ 
ment  of  whether  the  ability  to  draw 
conclusions  was  affected.  The  brief 
also  discussed  how  well  the  developer 
met  COMOPTEVFOR  requirements, 
and  then  provided  recommendations. 
COMOPTEVFOR  accredited  the  mod¬ 
els  and  simulations  when  convinced 
that  the  program  executive  office/ 
COMOPTEVFOR/developer  working 
team  had  satisfactorily  executed  its 
charter. 

The  operational  test  director  was  now 
able  to  complete  the  test  plan,  obtain 
its  approval  from  the  appropriate 
authority,  and  conduct  the  opera¬ 
tional  test.  Afterwards,  the  data  analy¬ 
sis,  final  report,  and  test  results  brief¬ 
ings  relied  heavily  upon  the 
verification,  validation,  and  accredita¬ 
tion  effort. 

Future  Challenges 

The  successful  achievement  of  certifi¬ 
cation  and  accreditation  for  the  opera¬ 
tional  test  did  not  mean  the  end  of  the 
SPY-ID(V)  validation,  verification,  and 
accreditation  process.  As  expected,  the 
subsequent  briefings  provided  to  PEO 
SC-AP,  COMOPTEVFOR,  and  the 
Director,  Operational  Test  and  Evalua¬ 
tion  resulted  in  feedback.  Thus,  some 
new  challenges  arose: 


•  Expand  existing  databases  by 
collecting  new  empirical  real- 
world  data. 

•  Refine  models  and  simulations 
fidelity,  such  as  the  sea  clutter 
mathematical  model,  to  more 
closely  approximate  real  sea 
clutter. 

•  Increase  the  capabilities  of 
essential  models  and  simula¬ 
tions,  such  as  incorporating  a 
fluctuating  radar  cross-section 
behavior  in  the  simulated  tar¬ 
gets. 

•  Overcome  certain  limitations, 
such  as  the  sea  clutter  genera¬ 
tor’s  inability  to  implement 
fully  the  sea  clutter  model. 

•  Improve  the  verification,  valida¬ 
tion,  and  accreditation  process. 

•  Investigate  new  models  and 
simulations  that  will  add  value 
to  future  developmental  and 
operational  tests. 

Lessons  Learned 

In  reality,  the  functioning  of  the  verifi¬ 
cation,  validation,  and  accreditation 
process  was  not  nearly  as  clean  or  lin¬ 
ear  as  oudined  in  this  article.  In  some 
cases,  the  developer  wrote  the  Simula¬ 
tion  Validation  Plan  and  the  Simula¬ 
tion  Validation  Report  concurrently; 
for  example,  if  a  validation  procedure 
proved  impractical  halfway  through, 
and  another  method  had  to  be  imple¬ 
mented.  In  other  cases,  a  model’s  veri¬ 
fication  and  validation  yielded  an 
unexpected  result.  Once  we  found 
that  a  model  intended  for  use  dis¬ 
played  an  undesired,  less-realistic 
effect  when  compared  to  other  indus¬ 
try  models.  Ultimately,  we  discarded 
this  model  and  selected  a  substitute. 
For  reasons  like  these,  the  SCP  was 
educational  for  all  its  members. 

We  continued  to  assimilate  lessons 
learned  throughout  the  course  of  this 
verification,  validation,  and  accreditation 
process.  A  brief  description  and  solution 
for  three  of  these  lessons  follow: 
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Lesson  1 

We  originally  constructed  the  SCP  as  a 
voting  body,  similar  in  makeup  to  the 
SMB.  However,  at  this  level  a  simple 
majority  vote  consisting  of  the  three 
developers  and/or  a  supporting  orga¬ 
nization  could  theoretically  override 
the  desires  of  either  the  program  exec¬ 
utive  office  or  COMOPTEVFOR.  The 
SMP  had  obligated  the  program 
executive  office  chairperson  and 
COMOPTEVFOR  co-chairperson  to 
support  mutually  the  plan’s  common 
goals.  For  either  individual  to  proceed 
without  the  complete  concurrence  of 
the  other  was  self-defeating,  regardless 
of  developers’  positions.  So  in  prac¬ 
tice,  voting  was  irrelevant  and  ulti¬ 
mately  eliminated;  a  simple  agreement 
between  chair  and  co-chair  moved  the 
SCP  forward. 

Lesson  2 

Only  one  SCP  existed  for  all  seven 
models  and  simulations.  The  Toma¬ 
hawk  Program’s  original  concept  of 
one  SCP  per  model  was  good,  but 
considered  impractical  for  SPY-ID(V) 
because  of  time  and  money  con¬ 
straints.  So,  each  SCP  meeting  ad¬ 
dressed  all  the  concerns  and  problems 
associated  with  each  model  and  simu¬ 
lation.  As  test  time  drew  near,  with 
much  left  to  do,  this  “do-everything-at- 
SCP-meeting”  approach  failed.  The 
SCP  could  not  efficiently  handle  all 
the  requirements  of  Simulation  Valida¬ 
tion  Report  development  for  seven 
models.  Simulation  Validation  Plan 
writing  turned  out  to  be  much  more 
challenging  and  controversial  than 
anticipated.  The  SCP  eventually  be¬ 
came  so  inundated,  a  permanent  ses¬ 
sion  appeared  necessary. 

The  solution  was  to  break  up  the  SCP 
into  smaller  teams  that  each  dealt  with 
a  subset  of  Simulation  Validation 
Reports.  This  allowed  the  available 
expertise  to  focus  more  completely 
and  exactly  than  before.  One  team’s 
membership  consisted  of  two  Lock¬ 
heed  Martin  experts  as  well  as  repre¬ 
sentatives  from  the  Naval  Surface  War¬ 
fare  Center  and  Center  for  Naval 
Analyses.  Another  team  included  an 
AEGIS  Technical  Representative 


staffer,  a  Lockheed  Martin  engineer, 
and  an  OPTEVFOR  analyst.  Represen¬ 
tation  on  each  team  also  included 
the  program  executive  office  and 
COMOPTEVFOR.  When  a  team 
wished  to  present  a  viable  product, 
the  membership  convened  the  formal 
SCP. 

Lesson  3 

The  honesty  and  integrity  of  all  the 
participants  in  the  verification  and  val¬ 
idation  process  was  absolutely  vital  to 
its  credibility.  The  co-chairing  offices 
hid  nothing  from  external  observers, 
including  some  rather  high-spirited 
controversies.  One  developer  immedi¬ 
ately  revealed  a  model’s  limitation, 
newly  discovered  during  verification 
and  validation,  that  impacted  unfavor¬ 
ably  on  its  use.  To  their  credit,  the 
supporting  activities  focused  their 
attention  on  problem  solving,  not  just 
problem  noting. 

Conclusion 

The  net  result  of  this  rather  involved 
process  had  several  positive  elements. 
All  parties  learned  that  a  model’s  lega¬ 
cy  is  not  sacrosanct.  We  uncovered 
preexisting,  unknown  capabilities  and 
limitations  that  led  to  a  more  precise 
use  of  the  models  and  simulations  and 
a  more  accurate  interpretation  of  test 
data.  Ultimately,  we  achieved  a  high 
degree  of  confidence  in  the  capabilities 
as  well  as  the  limitations  of  the  models 
and  simulations.  The  program  execu¬ 
tive  office  and  its  developers  al¬ 
so  gained  fresh  insight  about  their 
models  and  simulations  and  how  to 
improve  them. 

And  finally,  COMOPTEVFOR  authored 
an  operational  test  plan  that  realistical¬ 
ly  and  fairly  tested  the  radar  at 
CSEDS.  ASN(RDA)’s  acquisition  strat¬ 
egy  worked  as  intended,  and  the  Navy 
saved  a  lot  of  time  and  money.  Com¬ 
mon  sense  and  teamwork  made  this 
process  viable  and  successful  DoD 
will  see  more  of  these  efforts  in  future 
programs  as  the  program  office/devel- 
oper/operational  tester  combination 
works  smarter  to  place  the  best  tech¬ 
nology  available  in  the  hands  of  the 
warfighter. 
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MODELING  AND  SIMULATION 


Put  a  Virtual  Prototype 
on  Your  Desktop 


An  Air  Force  Collaborative  Research 
and  Engineering  Environment  for 
Acquisition  Reform 

WILLIAM  K.  MCQUAY 


■|  re  you  tired  of  reading  state- 
ments  of  work,  technical  spec- 
■M  ifications,  proposals,  and 
monthly  reports?  Have  you 
■^Dasked  yourself,  what  does  this 
proposal  really  mean?  What  is  the 
contractor  actually  saying,  or  more 
importantly,  what  will  the  deliverable 
really  be  able  to  do?  Or  perhaps  you’ve 
indulged  in  a  little  wishful  thinking:  If 
I  could  only  reach  out  and  touch  the 
new  system  before  it  exists  and  do  a 


Virtual  Reality  Battleroom  for  the  Joint  ... 
Synthetic  Battlhspace  -  a  "virtual  photo”  of 


IMMERSION  THEATRE  TO  DEMONSTRATE  FUTURE- 
TECHNOLOGY  AND  WEAPONS  SYSTEMS  USING  SIMU¬ 
LATION  AND  VISUALIZATION.  THE  PHOTO  IS  ACTUALLY 
A  DIGITAL  BsIHANGEMENT  OF  TWO  PHOTOS  DEPICT-  T 
ING  THE  INSIDE  OF  THE  DoD  WaRBREAKER  FACILITY 

•  in  Washington,  D:C;  the  faces  represent  peo- 

!  PLE  WHO  ACTUALLY  WORK  AT  WRIGHT  LABORATORY 


McQuay  is  Chief,  Simulation  Technology  Branch,  System  Concepts  and  Simulation  Division,  Avionics  Directorate,  U.S.  Air  Force  Wright  Laboratory,  Wright-Patterson 
Air  Force  Base,  Ohio.  Fie  directs  the  Electronic  Concepts  Simulation  Research  Laboratory  and  has  over  25  years'  experience  in  research  for  advanced  simulation 
technology.  McQuay  currently  chairs  an  Avionics  Directorate  Integrated  Product  Team,  which  is  defining  and  implementing  a  Collaborative  Engineering  Environ¬ 
ment  (CEE)  for  laboratory-wide  use  and  application  of  virtual  prototyping. 
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“virtual  test  drive”  now,  before  I  invest 
extensive  resources  in  their  concept. 
How  do  I  put  this  in  terms  that  all 
members  of  my  acquisition  team  can 
understand?  Under  Acquisition 
Reform,  as  a  program  manager  I  only 
have  insight  and  not  oversight  of  my 
contractor.  How  do  1  get  insight  into 
the  contractor’s  effort  when  I  have  less 
people  and  smaller  budgets? 

Help  Is  On  the  Way 

Good  news  —  help  is  on  the  way. 
Some  innovative  uses  of  simulation 
and  information  technologies  will 
bring  technical  and  program  manage¬ 
ment  data  in  a  comprehensible  format 
to  a  personal  computer  near  you: 
desktop  virtual  prototyping  and  col¬ 
laborative  engineering.  Changes  in 
simulation  and  information  technolo¬ 
gy  now  allow  computer  engineers  to 
create  computer  models  of  conceptual 
hardware  systems  prior  to  building  the 
actual  hardware.  The  collaborative 
development  of  a  digital  computer 
model  in  parallel  with  the  hardware  is 
called  Collaborative  Virtual  Prototyp- 


Changes  in 
simulation  and 
information 
technology  now 
allow  computer 
engineers  to  create 
computer  models 
of  conceptual 
hardware  systems 
prior  to  building 
the  actual 
hardware . 
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Engineers  and  analysts  wilL  usetheir  desk¬ 
top  PCs  AS  ACQUISmON  PORTALS  INTO  THE 
i Joint SYNTHEncBATriESPACE. During 

REQUIREMENTS  DEFINITION  PHASE,  THEY  WILL  BE  .  , 
IMMERSED  INTO  A  SYNTHETIC  ENVIRONMENT  —  A  -  : 

;  TWO- OR  THREE-DIMENSIONAL  WARGAME  WHERE  ; - 
THE  MILITARY  WORTH  OF  THE  PROPOSED  CONCEPT 
■  CAN  BE  EVALUATED  WTTH  REALISTIC  SCENARIOS  CT.' 
AND  LOCALES.  ' 

ing  (CVP).  Any  definition  of  CVP  must 
encompass  all  of  the  following  charac¬ 
teristics: 

CVP  is  the  application  oj  advanced 
information  systems  technology  in 
design ,  modeling ,  simulation,  analysis , 
manufacturing ,  testing,  and  logistics  to 
support  life-cycle  development  of  a  sys¬ 
tem  in  a  geographically  distributed 
electronic  environment. 

Its  use  throughout  DoD  is  consistent 
with  current  acquisition  trends  in  the 
Department  as  well  as  the  commercial 
sector  (Figure  1). 

Acquisition  Reform  and  the 
Joint  Synthetic  Battlespace  — 
Hade  Personal 

DoD  has  implemented  significant 
changes  in  how  it  buys  weapon  sys¬ 
tems.  The  new  emphasis  is  on  concur¬ 
rent  engineering  with  Integrated  Prod¬ 
uct  and  Process  Development  (IPPD) 
and  collaboration  with  Integrated 
Product  Teams  (IPT).  The  new  DoD 
vision  includes  Simulation  Based 
Acquisition,  a  process  supported  by 
robust,  collaborative  use  of  simulation 
technology  that  is  integrated  across 
acquisition  phases  and  programs. 

To  be  competitive  in  their  fields, 
throughout  the  commercial  sector 
world-class  companies  in  the  automo¬ 
tive,  electronics,  aircraft,  and  heavy 
equipment  manufacturing  areas  use 
CVP  and  collaborative  engineering  for 
requirements,  analysis,  and  design. 
You,  as  a  program  manager,  will  be 
working  with  companies  that  use 
these  technologies  to  design  their 
products.  As  partners  in  developing 
DoD  products,  these  companies  will 
be  applying  the  best  industry 
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FIGURE  1 .  Simulation  Based  Acquisition 
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practices  to  your  work,  and  you  will 
need  to  collaborate  with  them. 

Today,  a  commercial-sector  program 
manager  can  turn  on  a  personal  com¬ 
puter  (PC)  on  the  desktop,  check  E- 
mail,  and  then  look  at  the  status  of  the 
program,  -  a  completely  paperless, 
electronic  review.  That  same  program 
manager  can  distribute  solicitations 
electronically,  and  receive  return  pro¬ 
posals  by  the  same  mode.  Along  with 
the  standard  full  text  descriptions  of 
the  technical  task  in  their  return  pro¬ 
posals,  contractors  can  also  submit  a 
digital  model  of  the  concept  or  design. 

The  program  manager’s  technical  eval¬ 
uation  team  can  look  at  an  electronic 
representation  of  the  proposal  in  the 
form  of  a  computer  model.  The  model 
then  becomes  part  of  an  electronic 
design  and  a  simulatable  specification 
for  the  system.  Further,  the  technical 
team  can  also  “what  if”  —  hypothesize 
uses  of  the  system  and  run  excursions 
on  competing  versions  of  the  same 
concept  or  design. 


In  the  commercial  sector,  a  virtual  pro¬ 
totype  of  a  car  or  a  plane  allows  design 
teams  to  walk  through  the  virtual  pro¬ 
totype  to  see  how  the  components  are 
changing.  The  virtual  prototype  serves 
as  a  common  frame  of  reference  for 
the  designers,  engineers,  and  man¬ 
agers.  It  allows  you  as  the  program 
manager,  to  establish  a  level  playing 
field  for  consistent  comparisons 
among  alternative  concepts  and 
designs.  Ideally,  CVP  provides  the 
insight  you  need  into  what  your  con¬ 
tractor  is  doing. 

Even  earlier  in  the  acquisition  process, 
the  program  or  technical  manager  can 
work  with  the  user  to  define  require¬ 
ments  using  a  virtual  prototype.  His¬ 
torically,  program  requirements  are  dif¬ 
ficult  to  quantify  and  verbalize.  Users 
are  able  to  state  what  they  don’t  want 
much  easier  than  describing  what  they 
do  want.  A  simulation  model  devel¬ 
oped  in  parallel  with  the  hardware  or 
technology  development  allows  scien¬ 
tists,  engineers,  or  end  users  to  refine 
system  requirements  early  in  the  engi¬ 


neering  process.  The  users  then 
become  an  integral  part  of  the  design 
process.  Ultimately,  when  program 
managers  follow  IPPD  procedures  and 
bring  users  into  the  design  process, 
commercial-sector  applications  show  a 
significant  decrease  in  development 
time.  As  we  extend  this  approach  to 
military  acquisition,  the  Air  Force  Bat- 
tlelabs  will  allow  the  operational  com¬ 
mands  to  do  a  “virtual  test  drive”  of 
new  weapon  concepts  and  provide 
feedback  to  the  acquisition  communi¬ 
ty- 

Within  the  Air  Force,  we  envision  an 
integrated,  common  modeling  and 
simulation  (M&S)  environment  that 
will  be  accessed  by  analysts,  warfight¬ 
ers,  developers,  and  testers  supporting 
the  range  of  Air  Force  tasks,  from 
determining  requirements  through 
conducting  operations.  The  key  con¬ 
cept  in  the  Air  Force  M&S  vision  is  the 
Joint  Synthetic  Battlespace  -  an  inte¬ 
grated  M&S  environment  where  simu¬ 
lations  extend  from  high-level  aggre¬ 
gate  models  to  detailed  engineering 
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models,  from  pilots  in  live  aircraft  and 
simulators  to  hardware  components 
and  laboratory  test  beds. 

Your  desktop  PC  will  be  your  acquisi¬ 
tion  portal  into  the  Joint  Synthetic  Bat- 
despace.  During  requirements  defini¬ 
tion  phase,  you  will  be  immersed  into 
a  synthetic  environment  —  a  two-  or 
three-dimensional  wargame  where  the 
military  worth  of  the  proposed  con¬ 
cept  can  be  evaluated  with  realistic 
scenarios  and  locales.  Such  a  system 
allows  the  user  to  selectively  choose 
the  level  of  detail  needed  for  the  task 
at  hand,  draw  on  distant  resources, 
and  easily  “plug-and-play”  computer 
simulations,  manned  simulators,  and 
live  hardware  to  create  any  needed 
simulation  environment.  Demonstra¬ 
tions  of  a  future  system’s  military 
worth  will  be  conducted  in  the  syn¬ 
thetic  environment  represented  by  the 
Joint  Battlespace.  More  than  just 
acquisition  —  analysts,  researchers, 
decision  makers,  and  warfighters  must 
be  able  to  wplug  in”  to  a  common  bat- 


despace  from  their  desks,  simulators, 
or  crew  stations  in  order  to  assess, 
develop,  train,  or  conduct  warfight¬ 
ing. 

Your  industry  counterpart  has  long 
been  driven  by  cost  as  the  bottom  line. 
Under  Acquisition  Reform,  DoD  will 

Your  desktop 
PC  will  be 
your  acquisition 
portal  into 
the  Joint 
Synthetic 
Battlespace. 

yy 


make  buy  decisions  on  life  cycle-cost 
performance  trade  studies  where  cost 
is  an  independent  variable.  The  future 
Air  Force  Collaborative  Engineering 
Environment  (CEE)  will  have  con¬ 
straint-based  analysis  tools  to  aid  in 
early,  high-level  concept  trade  studies 
for  cost  of  function  and  cost  of  perfor¬ 
mance  for  various  alternative  technolo¬ 
gies. 

A  virtual  prototype  allows  the  engineer 
to  see  the  impact  of  design  changes. 
Trade  studies  using  the  model  can 
then  be  performed  throughout  devel¬ 
opment  as  an  essential  part  of  the  sys¬ 
tems  engineering  process. 

A  Collaborative  Research 
and  Engineering  Environment 
Near  You 

Two  of  the  most  significant,  technolog¬ 
ically  advanced  programs  are  the 
Avionics  CEE  development  project 
being  conducted  at  the  Avionics  Direc¬ 
torate,  Wright  Laboratory  (Figure  2); 
and  the  advanced  research  underway 
at  the  Defense  Advanced  Research 
Projects  Agency  (DARPA)  Simulation 
Based  Design  (SBD)  program.  The 
Avionics  Directorate  has  initiated  a 
program  to  develop  and  exploit  collab¬ 
orative  engineering  technologies  and 
implement  a  CEE  to  enhance  produc¬ 
tivity  by  advancing  avionics  collabora¬ 
tive  virtual  prototyping  processes.  It 
will  build  on  the  significant  commer¬ 
cial  technology  base  existing  for  elec¬ 
tronic  systems  design,  DARPA s  SBD 
initiative,  and  other  commercial/ 
industry  information  and  modeling 
standards  and  best  practices. 

Collaborative  Engineering  and  Virtual 
Prototyping  is  the  application  of 
advanced  distributed  M&S  and  engi¬ 
neering  tools  in  an  integrated  environ¬ 
ment  to  support  technology  develop¬ 
ment,  system  design,  performance, 
cost,  and  producibility  trade-off  analy¬ 
ses  throughout  the  entire  product  and 
system  engineering  life  cycle.  As  such, 
it  enables  all  members  of  an  IPPD  to 
continuously  interact  through  elec¬ 
tronic  modeling  and  data  interchange; 
increases  insight  into  life-cycle  con¬ 
cerns;  permits  earlier  testing  and 


FIGURE  2  CEE  Built  on  the  DARPA  SBD  Framework 


Commercial  Tools  legacy  Engineering  Tools 
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experimentation  through  virtual  test 
ranges;  and  accelerates  physical  pro¬ 
duction  through  process  optimization 
using  virtual  factories. 

Additionally,  Collaborative  Engineer¬ 
ing  simulations,  with  integral  product 
and  process  models,  will  permit  engi¬ 
neers  to  obtain  detailed  knowledge 
earlier  in  the  conceptual  and  prelimi¬ 
nary  design  phases  where  it  can  have 
the  most  influence  on  life-cycle  cost. 
More  emphasis  will  be  placed  on  the 
collaborative  development  of  virtual 
prototypes  of  key  technology  products 
to  demonstrate  their  military  effective¬ 
ness  and  worth  in  an  integrated  sys¬ 
tems/mission  environment 

As  downsizing  trends  continue  in 
both  defense  and  industry,  the  mili¬ 
tary  and  commercial  laboratories  will 
increasingly  depend  on  other  organi¬ 
zations  for  key  technologies  to  inte¬ 
grate  into  systems.  Additionally, 
increasing  demands  will  be  placed  on 
technology  to  facilitate  more  efficient, 


effective  collaboration  of  widely  dis¬ 
persed  personnel  across  many  differ¬ 
ent  application  domains  in  order  to 
solve  complex  problems  and  accom¬ 
plish  difficult  tasks. 

As  an  initial  response,  CVP  meets  the 
demand  for  technical  assistance  and 
provides  the  infrastructure  to  support 
these  new  acquisition  requirements.  It 
will  also  assist  in  the  breakdown  of 
technology  stovepipes  and  become  the 
construct  for  communication  of  tech¬ 
nologies  between  domains. 

CVP  can  be  implemented  in  many 
organizational  structures.  Traditional 
hierarchical  workplaces,  concurrent 
engineering  environments,  and  work 
groups  focused  on  rapid  prototyping 
are  a  few  examples.  Implementation  of 
a  CVP  system  requires  attention  to  the 
necessary  enabling  technologies  and 
supporting  infrastructure.  A  crucial 
part  of  a  CVP  system  implementation 
is  educating  personnel  on  how  CVP 
can  meet  customer,  organizational, 


and  individual  goals  as  well  as 
decrease  time-to-market,  lower  life-cycle 
costs,  and  improve  product  quality 

Historically,  80  percent  of  the  develop¬ 
ment  costs  and  70  percent  of  a  prod¬ 
uct’s  life-cycle  cost  are  determined 
during  conceptual  design.  As  the  pro¬ 
gram  moves  from  conceptual  design 
into  engineering  and  manufacturing 
development,  the  ability  to  substantial¬ 
ly  influence  life-cycle  costs  diminishes. 
The  freedom  to  make  design  changes 
decreases  as  the  knowledge  about  the 
system  design  increases.  In  other 
words,  a  progression  from  soft  to  hard 
information  occurs  as  the  system 
moves  from  the  conceptual  phase  to 
the  detailed  design  phase. 

CVP  can  move  the  knowledge  curve  to 
the  left  and  increase  the  hard  informa¬ 
tion  available  in  the  early  stages  of 
design.  This  improvement  in  the  quali¬ 
ty  of  information  should  benefit 
the  acceleration  of  the  technology  mat¬ 
uration  and  ultimately  facilitate 


FIGURE  3.  Multiple  Views  in  Collaboration 
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technology  transition.  The  end  result 
should  be  designs  completed  in  less 
time  and  at  less  cost. 

The  use  of  M&S  in  the  design,  devel¬ 
opment,  and  distribution  of  products 
is  not  a  new  concept  or  idea.  The  DoD 
and  industry  have  been  using  virtual 
prototyping  within  many  of  their  indi¬ 
vidual  functional  departments  and 
organizations  for  many  years.  Howev¬ 
er,  these  individual  stovepipe  groups  of 
functionality  have  not  interacted  with 
each  other  in  an  effective  way  and  have 
oftentimes  duplicated  functionality. 

A  CVP  system  provides  the  capability 
to  integrate  stovepipe  resources  and 
increase  the  collaborative  interactions 
of  the  people  using  the  resources. 
Thus,  the  old  mindset  of  having  to 
move  resources  needed  to  do  a  partic¬ 
ular  job  local  to  one  location  is  no 
longer  necessary  or  valid. 

In  the  future,  clusters  of  geographical¬ 
ly  separated  resources  will  be  integrat¬ 
ed  by  advanced  communications  net¬ 
works  into  a  virtual  system.  Users  will 
search  repositories  for  the  resources 
needed  to  solve  their  particular  appli¬ 
cation,  will  assemble  and  configure 
the  resources  into  a  virtual  system, 
and  will  execute  or  use  the  virtual  sys¬ 
tem  to  solve  their  problem  or  accom¬ 
plish  their  task.  Additionally,  products 
resulting  from  one  task  will  seamlessly 
interact  with  the  products  of  other 
tasks  to  accomplish  unique  functions. 

The  Collaborative  Research  and  Engi¬ 
neering  Environment  will  emphasize 
product  and  process  models.  Product 
and  process  model  applications  cap¬ 
ture  and  provide  information  about  a 
product  technology  development 
process. 

Product  Models.  These  models  pro¬ 
vide  details  about  the  specifications 
and  requirements  of  a  product,  its 
structure  and  behavioral  characteris¬ 
tics,  its  design  and  development  con¬ 
straint  rules,  and  the  different  versions 
of  the  design  and  implementation.  In 
this  context,  a  product  can  be  a  proto¬ 
type  piece  of  hardware,  a  report,  or  an 
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experiment/session.  Product  models 
also  define  any  special  test  equipment 
or  facilities  required  to  support  design 
and/or  development.  For  CVP,  the 
product  models  will  have  a  virtual  pro¬ 
totype  as  the  central  focus  of  all  other 
information  gathered  and  collected. 

Process  Models.  While  product  mod¬ 
els  focus  on  all  aspects  of  the  product 
design  and  development,  process 
models  provide  detailed  definitions  of 
the  engineering,  development,  and 
evaluation  processes  used  to  design 
and  develop  the  product.  Specifically, 
process  models  provide  information 
and  knowledge  on  how  to  use  various 
tools  and  resources  to  perform  the 
numerous  scientific,  engineering, 
development,  and  evaluation  tasks 


associated  with  technology  and  prod¬ 
uct  development. 

Making  Collaboration  Work  for 
Each  Team  Member 

Each  IPT  is  made  of  many  participants 
with  different  backgrounds,  experi¬ 
ences,  and  specialties.  They  literally 
do  not  speak  the  same  language.  The 
Collaborative  Research  and  Engineer¬ 
ing  Environment  must  provide  a 
domain-specific  view  in  the  native  ter¬ 
minology  of  each  of  your  team  partici¬ 
pants.  There  will  be  multiple  user 
interfaces  as  shown  in  Figure  3.  For 
example,  the  engineers  on  the  IPT 
must  be  able  to  employ  the  applica¬ 
tions  that  they  customarily  use.  The 
engineering  user  interface  must  be 
intuitive  for  the  engineering  domain. 
Similarly,  the  manufacturing,  financial, 
logistics,  management,  and  end  user 
must  be  able  to  access  the  informa¬ 
tion,  databases,  and  virtual  prototypes 
in  a  fashion  natural  to  their  way  of 
doing  business. 

The  overall  architecture  for  the  CEE  is 
a  layered,  open-systems  approach.  The 
infrastructure  consists  of  that  hard¬ 
ware  and  software  which  provides 
functionality  to  the  user,  but  resides  in 
the  background  and  does  not  directly 
interact  with  the  user.  The  user  sees  a 
consistent  interface  that  is  based  on 
Web  technologies  that  provide  porta¬ 
bility  to  many  different  platforms, 
including  the  workhorse  PC  on  your 
desktop. 

CEE/CVP — Crucial  Ingredients 

Advances  in  software  and  computer 
technology  are  making  desktop  CVP 
possible  and  affordable  for  the  engi¬ 
neering  process  in  government  and 
industry  research.  CVP  will  become  a 
crucial  means  of  sharing  technology 
and  systems  integration  for  research 
and  development  and  is  a  natural 
extension  of  the  Air  Force  vision  for  an 
integrated,  common  M&S  environ¬ 
ment,  accessed  by  analysts,  re¬ 
searchers,  warfighters,  developers,  and 
testers.  Virtual  prototyping  and  a  CEE 
are  crucial  ingredients  for  Acquisition 
Reform  —  providing  insight  for  the 
program  manager. 
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MODELING  AND  SIMULATION 


The  Theater  Missile 
Defense  System  Exerciser 

THOSE  -  Build  a  Little,  Test  a  Little 

LT.  COL.  STEVE  MCQUEEN,  U.S.  AIR  FORCE 
RAYMOND  B.  WASHBURN,  P.E.  *  JOHN  F.  MORASH 


Theater  ballistic  missiles  and 
cruise  missiles  are  a  major 
threat  to  U.S.  forces  deployed 
almost  anywhere  in  the  world. 
To  counter  this  threat,  an 
extremely  sophisticated  family  of  the¬ 
ater  missile  defense  (TMD)  weapon 
systems  has  been  developed.  To 
achieve  the  maximum  firepower  effec¬ 
tiveness,  however,  today’s  TMD  Family 
of  Systems  (FoS)  must  be  highly  inter¬ 
operable  to  counter  a  broad  spectrum 
of  threats,  environments,  and  deploy¬ 
ment  scenarios. 

The  Theater  Missile  Defense  System 
Exerciser,  or  TMDSE,  offers  the  only 
hardware-in-the-loop  (HWIL)  test 
capability  available  to  integrate  the 
entire  TMD  FoS  and  test  interoperabil¬ 
ity  issues  that  exist  between  the  sepa¬ 
rately  developed  TMD  systems. 

The  TMDSE  is  a  computer-based  test 
tool  used  to  verify  interoperability 
between  geographically  distributed 
TMD  systems  and  sensors.  This  tool 
“drives”  tactical  TMD  weapon  systems 
with  a  time-synchronized  simulated 
environment,  including  threats  (the¬ 
ater  ballistic  missiles,  cruise  missiles, 
and  aircraft),  weather,  and  terrain. 

In  June  of  1994,  the  U.S.  Army  Pro¬ 
gram  Executive  Office  for  Air  and  Mis¬ 


sile  Defense  (PEO  AMD)  located  in 
Huntsville,  Ala.,  successfully  conduct¬ 
ed  a  Proof-of-Principle  (POP)  demon¬ 
stration  of  a  test  tool  concept  that 
would  later  become  the  TMDSE.  This 
POP  demonstration,  that  validated  the 
concept,  illustrated  the  interconnec¬ 
tion  of  two  remote  TMD  tactical  hard¬ 
ware  sites  (the  U.S.  Army  PATRIOT 
Flight  Mission  Simulator  [FMS]  in  Bed¬ 
ford,  Mass.;  and  the  U.S.  Army  Joint 
Tactical  Ground  Station  [JTAGS]  locat¬ 
ed  in  Azusa,  Calif.),  simultaneously 
driven  in  real  time  with  a  common 
theater  test  environment. 

Following  the  TMDSE  POP,  the  Ballis¬ 
tic  Missile  Defense  Organization 
(BMDO)  then  directed  that  PEO  AMD 
in  Huntsville,  Ala.,  develop  the 
TMDSE,  thereby  providing  the  capa¬ 
bility  to  verify  that  the  TMD  FoS  are 
integrated  and  can  effectively  interop¬ 
erate  across  the  spectrum  of  threats, 
environments,  deployments,  and  con¬ 
tingencies  that  are  delineated  in  their 
respective  operational  requirements 
documents. 

Under  the  direction  of  the  Deputy  for 
Acquisition/Theater  Missile  Defense, 
BMDO,  TMDSE  development  is  in  its 
third  year  and  proceeding  to  an 
enhanced  Build  2  capability.  Ar  Force 
Lt.  Col.  Steve  McQueen,  BMDO/AQI, 


Systems  Integration/BMC3,  is  the  Pro¬ 
gram  Integrator.  As  executing  agent  for 
BMDO,  PEO  AMD  is  responsible  for 
the  development  of  the  TMDSE  Con¬ 
trol  Segment,  development  of  the 
Army  “drivers,”  and  integration  of  all 
Joint  elements. 

The  TMD  systems  that  are  integrated 
into  TMDSE  will  be  combinations  of 
existing  inventory,  product  upgrades, 
and  new  systems  that  evolve  to 
enhance  mission  effectiveness.  Its 
phased,  incremental  development 
approach  also  allows  TMDSE  to  be 
systematically  upgraded  to  higher  lev¬ 
els  of  fidelity  and  complexity  to  sup¬ 
port  the  evolving  TMD  architecture 
and  its  resulting  test  needs. 

As  the  complexity  of  the  deployable 
TMD  Systems  and  their  operating 
environments  increases,  so  must  the 
capability  of  the  TMDSE.  The  imple¬ 
mentation  of  BMDO’s  direction  will 
be  accomplished  through  the  phased 
development  of  the  TMDSE.  Each 
phase  during  this  development  pro¬ 
gression  is  referred  to  as  a  Build. 

TIIDSE  Build  1  Configuration 

TMDSE’s  developers,  Nichols  Re¬ 
search  Corporation  and  Teledyne 
Brown  Engineering,  of  Huntsville,  Ala., 
completed  the  TMDSE  Build  1  config- 
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uration  in  April  1996.  Upon  comple¬ 
tion,  the  TMDSE  Build  1  had  200,000 
lines  of  Ada  code.  The  TMDSE  Builds, 
leveraging  heavily  from  the  various 
Major  Defense  Acquisition  Programs 
and  other  defense  systems  that  make 
up  the  TMD  FoS,  make  TMDSE  a  very 
cost-effective  HW1L  test  capability.  As 
configured,  TMDSE  interfaces  direcdy 
with  each  weapon  system  via  its  exist¬ 
ing  tactical  driver,  and  does  not 
require  co-location  of  test  articles.  Five 
sites  jointly  participated  in  the  TMDSE 
Build  1  configuration: 

•  U.S.  Army  PATRIOT  Engagement 
Control  Station  (ECS)  and  Informa¬ 
tion  Control  Center  (ICC)  at  the 
U.S.  Army’s  Missile  Command 
(MICOM)  Software  Engineering 
Directorate,  Huntsville,  Ala. 

•  U.S.  Navy  AEGIS  Weapon  System  at 
the  Naval  Surface  Warfare  Center 
(NSWC)  AEGIS  Computer  Center 
(ACC)  at  Dahlgren,  Va. 

•  U.S.  Army  Joint  Tactical  Ground  Sta¬ 
tion  (JTAGS),PEO  AMD,  Huntsville, 
Ala.  (The  actual  JTAGS  shelter  driver 
was  and  is  housed  at  Aerojet  Corpo¬ 
ration  in  Azusa,  Calif.) 


The  TMDSE  is  a  computer 
based  test  tool  used  to 


verify  interoperability 
between  geographically 
distributed  HMD  systems 


and  sensors.  TMD  systems 
and  sensors.  This  tool 
“drives”  tactical  TMD 
weapon  systems  with  a 
time-synchronized 
simulated  environment, 
including  threats  (theater 
ballistic  missiles,  cruise 
missiles^iiidairawfl), 
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weather,  and  terrain. 


•  U.S.  Air  Force  SHIELD  at  the  Joint 
National  Test  Facility  QNTF),  Falcon 
Air  Force  Base,  Colorado  Springs, 
Colo. 

•  U.S.  Air  Force  Control  and  Report¬ 
ing  Center  (CRC)  at  the  Theater  Air 
Command  and  Control  Simulation 
Facility  (TACCSF),  Kirtland  Air 
Force  Base,  Albuquerque,  N.M. 

One  of  the  things  that  separates  the 
TMDSE  from  other  simulations  and 
contributes  to  its  uniqueness  is  its  use 
of  real  tactical  hardware  and  real  tacti¬ 
cal  communications.  During  actual 
TMDSE  execution,  the  TMDSE  makes 
use  of  a  real  PATRIOT  ICC  and  real 
PATRIOT  TMDSE  Control  Segment 
shelters,  real  AEGIS  weapon  system 
computers  and  software,  real  JTAGS 
computers  and  software,  and  real 
satellite  broadcasts.  (The  simulated 
threat  “injected”  into  the  JTAGS  and 
SHIELD  systems  will  generate  real 
Tactical  Information  Broadcast  Service 
[TIBS]  and  TRAP  Data  Distribution 
System  [TDDS]  cueing  messages  that 
will  be  received  by  the  PATRIOT, 
AEGIS  Weapon  System,  and  CRC  ele¬ 
ments.) 


The  Theater  Missile  Defense  System  Exerciser  or TMDSE  offers  the  only  tactical  Hardwarehn-the-Loop  (HWIL)  test  capability  available  to 

INTEGRATE  THE  ENTIRE  THEATER  MlSSILE  DEFENSE  FAMILY  OF  SYSTEMS  (TMD  FoS  )  AND  TEST  INTEROPERABILITY  ISSUES  THAT  EXIST  BETWEEN  THE  SEPARATELY 
DEVELOPED  TMD  SYSTEMS.  ' 
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The  TMDSE  system  is  connected  to  its 
remote  sites  using  two  separate  net¬ 
works:  one  that  addresses  the  test  con¬ 
trol  functionality  of  the  system,  and 
the  other  that  provides  the  tactical 
communications  network  for  the  sys¬ 
tems  under  test.  These  communica¬ 
tion  networks  consist  of  a  combina¬ 
tion  of  local  and  wide  area  networks, 
high  bandwidth  (i.e.,  T1  1.544 
megabit  per  second)  telephone  lines, 
KG- 194  encryption  devices,  and 
secure  telephones  (STU-IIIs),  which 
connect  the  TBE  Test  Exercise  Con¬ 
troller  (TEC)  hub  to  the  geographical¬ 
ly  distributed  TMD  Tactical  Drivers. 

The  first  of  these  is  the  TMDSE  test 
control  network,  which  is  comprised 
of  high  band  width  (Tl)  encrypted 
telephone  lines  that  join  the  TEC  with 
all  Remote  Environments  at  each  Tac¬ 
tical  Driver  site.  This  network  provides 
a  common,  synchronized  environment 
to  the  various  tactical  systems  via  a 
common  standardized  set  of  Distrib¬ 
uted  Interactive  Simulation  (DIS)  pro¬ 
tocol  data  units  (PDU).  Using  DIS 
PDUs,  TMDSE  injects  a  real-time, 
common  threat  scenario  into  real,  geo¬ 
graphically  distributed  tactical  sensors 
and  weapon  systems.  The  tactical  sys¬ 
tems  respond  in  real  time  via  their 
respective  tactical  communication 
data  nets,  including  TIBS/TDDS  and 
the  Joint  Data  Net,  allowing  each 
individual  TMD  system  to  operate 
synergistically  in  a  tactically  realistic 
battlefield. 

This  test  control  network  allows  the 
TMDSE  to  — 

•  generate  realistic  scenarios,  includ¬ 
ing  natural  (weather  and  terrain) 
and  artificial  environments,  includ¬ 
ing  tactical  missiles  and  air-breath¬ 
ing  threats; 

•  generate  realistic  missile  interceptor 
flyouts; 

•  generate  realistic  interceptor  and 
threat  debris  in  real  time; 

•  coordinate  and  synchronize  the 
stimulation  of  the  track  processing 
systems;  and 

•  coordinate  and  synchronize  dynam¬ 
ic  events  that  are  a  result  of  offen¬ 


sive/defensive  actions.  (“Dynamic” 
events,  as  opposed  to  “scripted” 
events  such  as  tactical  missile  fly¬ 
outs,  are  the  defensive  actions  taken 
by  the  tested  weapon  systems  in 
response  to  the  scripted  threats.  For 
example,  the  reaction(s)  of  a  PATRI¬ 
OT  fire  unit  to  approaching  tactical 
missiles  or  aircraft  must  be  repre¬ 
sented  dynamically  in  real  time.) 

The  second  network  used,  the  tactical 
communications  network,  connects 
the  tactical  systems  to  each  other. 
These  interfaces  must  appear  to  be  the 
natural  communications  expected  of 
the  TMD  components  with  regard  to 
protocol,  message  formatting,  and 
routing  selection.  Actual  Joint  Tactical 
Information  Distribution  System 
(JTIDS)  radio  terminals  cost  approxi¬ 
mately  $1  million  each  and  operate  via 
line-of-sight,  which  means  that  they 
are  restricted  to  distances  of  30-50 
kilometers.  Due  to  the  high  cost  of 
these  radios  and  the  fact  that  geo¬ 
graphically  distributed  TMDSE  sys¬ 
tems  are  sometimes  separated  by  dis¬ 
tances  of  hundreds  or  thousands  of 
miles,  another  means  had  to  be  found 
to  emulate  tactical  communications. 

For  TMDSE,  the  U.S.  Naval  Com¬ 
mand,  Control,  and  Ocean  Surveil¬ 
lance  Centers  Link  16  Emulator  and 
Communications  Monitor  (the  “NRaD 
Gateway”)  provided  the  tactical  com¬ 
munication  link  connectivity  between 
the  individual  weapon  system  plat¬ 
forms  using  the  Tactical  Digital  Infor¬ 
mation  Link  (TADIL)  J  protocols  and 
message  formats  emulating  a  JTIDS. 
Future  planned  enhancements  to  the 
NRaD  Gateway  will  increase  the  fideli¬ 
ty  of  the  TMDSE  and  allow  land-line 
emulation  of  satellite  transmissions. 

In  the  first  quarter  of  fiscal  year  1997, 
the  TMDSE  Build  1  configuration  was 
installed  at  the  Joint  National  Test 
Facility  (JNTF)  located  at  Falcon  Air 
Force  Base  in  Colorado  Springs,  Colo. 
BMDO  designated  the  JNTF  to  be  the 
operational  facility  where  FoS  tests  will 
be  run.  PEO  AMD,  however,  will  con¬ 
tinue  as  the  developer  for  the  follow- 
on  configurations. 


TMDSE  Build  2  Configuration 

The  TMDSE  Build  2  is  scheduled  to  be 
completed  by  July  1997.  By  the  end  of 
third  quarter,  fiscal  year  1997,  the 
TMDSE  Build  2  requirements  and 
functional  capabilities  will  demon¬ 
strate  an  evolving  capability  for  TMD 
system  integration  and  interoperability 
testing.  In  addition  to  the  original  five 
TMDSE  Build  1  systems  (PATRIOT, 
AEGIS,  CRC,  JTAGS,  and  SHIELD), 
the  following  two  additional  TMD  Tac¬ 
tical  Systems  will  participate  in  the 
Build  2  configuration: 

*  U.S.  Army  Theater  High  Altitude 
Area  Defense  (THAAD)  weapon  sys¬ 
tem,  PEO  AMD,  Huntsville,  Ala. 

•  U.S.  Marine  Corps  HAWK  TPS-59 
radar  system,  USMC  Systems  Com¬ 
mand,  Syracuse,  N.Y. 

TMDSE  Verification,  Validation, 
and  Accreditation  (W&A) 

The  technical  difficulties  and  costs 
associated  with  ensuring  adequate  ver¬ 
ification  and  validation  (V&V)  of 
modeling  and  simulation  (M&S)  are 
major  challenges  in  successfully  exe¬ 
cuting  a  simulation  development 
enterprise  within  the  DoD.  In  today’s 
current  regulatory  environment,  DoD 
and  Service  policies  and  directives 
generally  mandate  that  M&S  be  sub¬ 
jected  to  a  formal,  structured  verifica¬ 
tion,  validation,  and  accreditation 
(W&A)  program.  Within  the  Services, 
and  BMDO  in  particular,  guidance 
and  oversight  for  M&S  W&A  is 
becoming  quite  explicit.  Timely  and 
successful  accreditation  of  simulations 
with  embedded  legacy  models  and 
codes,  such  as  TMDSE,  require  explic¬ 
it,  focused  V&V  evaluations  that  are 
tied  to  the  simulations’  intended  use. 

The  best  means  for  accomplishment  of 
this  complex  task  is  a  rigorous, 
focused  V&V  and  evaluation  effort, 
which  is  adaptable  to  the  particular 
unit-under-test.  Currently,  for  TMDSE 
a  tailored  V&V  program  is  being  pur¬ 
sued  that  is  based  on  — 

*  leveraging  ongoing,  system-level 
simulation  development,  test,  and 
V&V  activities; 
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tion  decision  by  potential  TMDSE  cution  by  the  respective  system  simu- 
users  and  operational  testers.  lation  activity. 


•  verifying  TMDSE  through  a  series 
of  well-defined  and  coordinated 
functional  configuration  audit 
activities; 

•  validating  TMDSE  at  the  system 
level  by  explicitly  linking  TMDSE 
validation  activities  to  existing, 
ongoing,  or  planned  system  test 
activities  as  the  principal  source  of 
“real  world”  data;  and 

•  generating  the  essential  information 
necessary  for  V&V  reports  and  find¬ 
ings,  which  provide  the  evidence 
required  to  support  the  accredita- 


The  set  of  specific  validation  activities 
selected  for  execution  are  being  close¬ 
ly  coordinated  with  the  individual  sys¬ 
tem  developers  and  will  be  based 
upon  TMDSE  accreditation  data 
needs,  the  realities  of  the  system  pro¬ 
grams,  and  the  fixed  resources  avail¬ 
able  for  TMDSE  V&V  within  the 
respective  Services  and  BMDO.  The 
validation  activities  for  TMDSE  are 
being  defined  by  the  sponsor  for  exe- 


Hardware-in-the-Loop 
Test  (HWILT) 

The  fiscal  year  1996  BMDO  Hardware- 
in-the-Loop  Test  (HWILT-96)  was  con¬ 
ducted  in  September  1996  using  the 
TMDSE  Build  1  software.  Navy  Cmdr. 
Don  Gold  of  BMDO  was  the  program 
integrator  for  the  HWILT-96.  The  test 
was  executed  and  controlled  under 
the  direction  of  Army  Lt.  Col.  Chuck 
Treece  of  PEO  AMD,  from  the  develop- 


PM  :  SEPTEMBER-OCTOBER  1997  103 


mental  TMDSE  Test  Exercise  Con¬ 
troller  located  at  Teledyne  Brown  Engi¬ 
neering  in  Huntsville,  Ala. 

The  HWILT-96  tactical  weapon  system 
participants  generated  and  distributed 
tactical  communication  messages, 
including  Joint  Data  Network,  TADIL- 
J,  and  live  TIBS  and  TDDS  broadcasts. 
Dedicated  TIBS  and  TDDS  exercise 
channels  were  used  by  TMDSE  during 
the  test  to  preclude  the  broadcast  of 
exercise  tactical  event  messages  into 
the  actual  scenario  theater’s  opera¬ 
tional  network.  Ongoing  analysis  of 
the  collected  data  is  currently  being 
conducted. 

For  the  HWILT-96,  a  northeast  Asia 
scenario,  including  a  dynamic  environ¬ 
ment  of  threats  (theater  ballistic  mis¬ 
siles,  aircraft,  and  cruise  missiles), 
interceptors,  weather,  terrain,  and 
threat/interceptor  fragment  debris  was 
injected  into  the  HWIL  tactical 
weapon  systems.  The  HWILT-96  test 
event  employed  real  tactical  TMD 
assets  and  operators,  communicating 
via  real-world  tactical  communication 
links  responding  in  real  time  as  if  in 
an  actual  batdefield  situation. 

Future  HWILTs  will  be  executed  and 
controlled  from  the  BMDO  JNTF,  Fal¬ 
con  Air  Force  Base,  Colorado  Springs, 
Colo.  Installation  of  the  TMDSE  Build 
1  capability  has  been  completed  at  the 
facility.  Upon  completion  and  demon¬ 
stration,  subsequent  TMDSE  builds 
will  be  installed  at  the  JNTF  for  the 
operational  execution  of  future  TMD 
FoS  tests. 

The  successful  execution  of  the 
F1WILT-96  enabled  the  establishment 
of  policies  and  procedures  for  direc¬ 
tion  and  conduct  of  future  FoS  tests, 
the  development  of  lessons  learned 
from  the  early  use  of  TMDSE  for  FoS 
testing  to  support  definition  of  future 
TMDSE  enhancements,  and  early 
insight  into  FoS  interoperability  with 
respect  to  selected  TMD  Command 
and  Control  (C2)  Plan  objectives.  This 
experience,  in  conjunction  with  the 
full  cooperation  between  BMDO,  the 
operational  test  agencies,  and  the  Ser¬ 


vices  will  make  TMDSE  the  tool  of 
choice  whenever  TMD  system  test  and 
evaluation  issues  are  addressed. 

Leveraged  Activities 

The  PEO-AMD  provided  critical  sup¬ 
port  to  Joint  Project  Optic  Cobra 
(JPOC)  ’96  and  Joint  Exercise  Roving 
Sands  ’96,  conducted  in  June  1996. 
JPOC  is  an  annual  U.S.  Central  Com¬ 
mand  TMD  exercise  supported  with 
BMDO  funding.  Conducted  in  the  Fort 
Bliss,  Texas,  and  White  Sands  Missile 
Range,  N.M.,  areas  as  a  part  of  the  U.S. 
Forces  Command-managed  Joint  Exer¬ 
cise  Roving  Sands,  JPOC  is  the  world’s 
largest  Joint  Tactical  Air  Operations 
exercise.  During  the  exercise,  PEO 
AMD  successfully  implemented  and 
executed  the  Cooperative  Air  and  Mis¬ 
sile  Defense  Network  (CAMDEN),  a 
distributed  interactive  simulation 
infrastructure  capability  that  provides 
an  integrated  tactical  missile  and  air¬ 
craft  training  environment  for  the  U.S. 
and  allied  soldiers,  sailors,  airmen,  and 
Marines  participating  in  the  Roving 
Sands  exercise.  Some  CAMDEN  com¬ 
ponents  were  derived  from  ongoing 
PEO  AMD  simulation  and  test  and 
evaluation  programs  funded  by 
BMDO,  the  Defense  Modeling  and 
Simulation  Office,  and  from  other  Ser¬ 
vice  and  Joint  programs.  Particularly 
noteworthy,  however,  are  the  TMDSE- 
developed  elements  that  include  the 
PATRIOT  Digital  Flight  Mission  Simu¬ 
lator,  the  JTAGS  simulator,  the  THAAD 
Test  Controller,  and  the  AEGIS 
weapon  system  at  NSWC. 

Summary 

With  declining  resources,  missile  flight 
test  costs  are  a  major  expense  to  pro¬ 
gram  offices.  Many  constraints  influ¬ 
ence  live  flight  tests  such  as  range 
restrictions,  treaty  limitations,  environ¬ 
mental  concerns,  and  range  safety 
issues.  Program  offices  are  no  longer 
able  to  conduct  the  number  of  flight 
tests  that  they  once  did.  A  single  flight 
test  can  cost  from  $25  to  $50  million 
when  target,  interceptor,  range,  and 
personnel  costs  are  figured  in.  In  addi¬ 
tion,  the  number  of  simultaneous 
engagements  per  test  is  limited  to 
probably  no  more  than  two.  However, 


weapon  system  interoperability  assess¬ 
ment  is  required  in  a  “target  enriched” 
environment.  For  these  reasons, 
HWIL  testing  is  becoming  increasingly 
important  due  to  the  significant  cost 
savings  that  can  be  achieved  by  its  use, 
and  the  TMDSE  is  being  viewed  as 
BMDO’s  key  FoS  test  tool  resource. 

TMDSE  is  more  economical  than  live 
flight  tests  and  allows  TMD  systems  to 
explore  interoperability  issues  into 
areas  not  possible  during  live  flight 
tests,  such  as  multiple,  simultaneous 
engagements  and  stressing  environ¬ 
ments.  Expanding  beyond  range  limi¬ 
tations  as  well  as  logistical  considera¬ 
tions,  TMDSE  provides  an  economic 
solution  to  live  flight  tests. 

The  TMDSE  is  an  integral  part  of 
BMDO’s  overall  test  and  evaluation 
strategy  that  supports  the  successful 
acquisition  of  the  TMD  FoS.  The 
strengths  of  the  TMDSE  include  its 
design  flexibility  that  facilitates  the 
incorporation  of  new  tactical  weapon 
system  elements  by  easily  interfacing 
these  elements  into  the  distributed, 
real-time  TMDSE  network.  As  the 
TMD  FoS  evolves,  the  TMDSE  will 
mature  to  meet  the  challenge  of 
assessing  the  interoperability  of  these 
deployed  weapon  systems. 

The  “build-a-little,  test-a-litde”  metho¬ 
dology  implemented  for  the  TMDSE 
will  reduce  development  risks,  pace 
the  program  to  the  funding  appropria¬ 
tions,  and  tailor  the  “builds”  to  the 
TMD  weapon  system  development 
schedules.  The  experience  of  the  PEO 
AMD  TMDSE  development  team  has 
provided  a  solid  foundation  to  lever¬ 
age  into  the  Build  3  development 
effort.  This  experience,  in  conjunction 
with  the  full  cooperation  between 
BMDO  and  the  Services,  will  make 
TMDSE  the  tool  of  choice  whenever 
TMD  system  test  and  evaluation  issues 
are  addressed. 

For  additional  information  on  the 
TMDSE  Program,  visit  http:// 
peoamd  .redstone .  army.mil/tmdse  / 
—  our  TMDSE  Home  Page  on  the 
World  Wide  Web. 
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DUAL 


USE  TECHNOLOGY 


A  New  Vision, 

Further  Leveraging  Emerge  From 
Orlando's  Simulation  Superstructure 

WTET  Prototype  Developed  By 
Collaboration,  Partnerships,  Cooperation 
Between  Government  and  Industry 


efense  capabilities  in  educa- 
j^^Btion  and  training  represent 
■  Ban  important  resource.  New 
programs  will  accelerate 
mU  transfer  of  this  experience 
to  civilian  institutions.  The  Depart¬ 
ment  of  Defense  and  NASA  [Nation¬ 
al  Aeronautics  and  Space  Adminis¬ 
tration}  have  invested  heavily,  both 
in  the  hardware  and  software  need¬ 
ed  for  advanced  instructional  sys¬ 
tems;  they  have  accumulated  valu¬ 
able  experience  in  how  to  use  the 
new  technologies  in  practical  teach¬ 
ing  situations.  The  Navy  Training 
Systems  Center  [now  the  Naval  Air 
Warfare  Center  Training  Systems 
Division]  and  the  Army  Simulation, 
Training,  and  Instrumentation 
Command  together  spend  about  $1 
billion  a  year  on  training  systems. 
There  are  over  150  defense  simula¬ 
tion  and  training  companies  serving 
these  needs  in  Central  Florida 
alone....”1 

—President  William  J.  Clinton 
Vice  President  Albert  Gore,  Jr. 

February  22,  1993 

From  the  nationally  recognized  simula¬ 
tion  superstructure  in  Orlando  comes 
a  new  vision  —  and  further  leveraging. 

The  acquisition  manager  of  today 
must  be  aware  of  alternative  vehicles, 
available  outside  of  the  Federal  Acqui- 


JEFFREY  D.  HOREY 

sition  Regulations,  which  can  be  used 
to  ensure  a  technologically  superior 
product,  produced  in  a  cost-effective 
manner  by  a  reliable  industrial  source. 

Weapons  Team 
Engagement  Drainer 

An  example  of  one  such  vehicle  is  the 
cooperative  agreement  among  the 
Naval  Air  Warfare  Center  Training  Sys¬ 
tems  Division  (NAWCTSD);  SBS  Tech¬ 
nologies,  Inc.;  and  Camber  Corpora¬ 
tion,  to  produce  the  Weapons  Team 
Engagement  Trainer  (WTET).  The 


Office  of  the  Secretary  of  Defense, 
under  the  Defense  Laboratory  Partner¬ 
ship  Program  for  Technology  Transfer, 
funds  the  agreement. 

The  WTET  is  an  advanced  Special 
Weapons  and  Tactics  (SWAT)  training 
system  that  allows  multiple  member 
weapon  teams  to  participate  in  multi¬ 
ple  room  (and  multiple  screen)  threat 
engagements,  under  shootback  and 
advanced  individual  and  team  perfor¬ 
mance  feedback  conditions.  NAW¬ 
CTSD  initially  developed  the  system. 


Horey  is  currently  a  Project  Director,  Naval  Air  Warfare  Center  Training  Systems  Division  (NAWCTSD),  Orlando,  Fla.  He  holds  an  MS.  in  Industrial  Psychology  from 
The  George  Washington  University  For  the  past  10  years,  he  has  worked  in  the  areas  of  training  assessment,  design,  and  evaluation  for  NAWCTSD. 
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A  prototype  of  the  WTET  was  exten¬ 
sively  and  successfully  demonstrated 
to  law  enforcement  agencies  and  spe¬ 
cial  operations  groups  of  the  U.S.  mili¬ 
tary  The  enhanced  production  version 
will  be  demonstrated  in  1997.  It  will 
provide  instructor-controlled  training 
and  feedback  for  a  wide  range  of  law 
enforcement  and  military  threat  situa¬ 
tions.  Included  in  the  system  will  be 
the  training  capability  for  use  of  force 
decision  making;  marksmanship  skills 
and  analysis;  SWAT  operations, 
including  sniper  training;  and  use  of 
less-than-lethal-force  weapons. 

Industry  and  the 
Commercialization  Process 

The  industry  partner,  SBS  Technolo¬ 
gies,  Inc.,  already  produces  a  judgmen¬ 
tal  use-of-force  trainer,  for  both  the 
law  enforcement  and  military  commu¬ 
nities.  Under  the  commercialization  of 
WTET,  the  merging  of  their  current 
trainer  and  the  many  unique  features 
of  WTET  will  result  in  a  training  sys¬ 
tem  that  will  provide  a  full  and  com¬ 
plete  range  of  weapons,  team,  and 
engagement  training  under  realistic 
tactical  situations. 

This  is  the  first  use  of  a  cooperative 
agreement  for  commercialization  with¬ 
in  the  Naval  Air  Systems  Command, 
NAWCTSD’s  parent  organization.  As 
such,  WTET  has  been  designated  as  a 
pilot  project. 

Authority  to  use  the  legal  vehicle 
selected  for  this  commercialization 
process  —  the  cooperative  agreement 
—  was  recently  granted  to  the  military 
services. 

The  commercialization  process  con¬ 
sists  of  a  two-year  cycle  of  system 
development  by  NAWCTSD  and  its 
industry  partners,  along  with  the 
direct  involvement  of  the  user  commu¬ 
nity  Traditional  programmatic  reviews 
are  ensured  during  the  life  of  this  non- 
traditional  technology  transfer  project. 
The  program  management,  engineer¬ 
ing  oversight,  and  training  require¬ 
ment  functions  during  the  commer¬ 
cialization  are  being  performed  by 
NAWCTSD. 


User  Community 

Interested  user  agencies  also  will  be 
integrated  into  the  effort  to  ensure  the 
final  product  reflects  the  requirements 
of  the  military  and  civilian  law 
enforcement  communities  (federal, 
state,  and  local).  As  part  of  the  pro¬ 
gram  plan,  two  systems  will  be  avail¬ 
able  for  evaluation  by  those  communi¬ 
ties. 

Sponsored  in  part  by  the  National 
Institute  of  Justice,  the  initial  system 
installation  has  been  designated  for 
the  Los  Angeles  County  Sheriff’s 
Department  Laser  Village  Training 
Facility.  Ideally,  it  should  be  opera¬ 
tional  by  the  end  of  1997,  and  will  be 
available  for  use  by  military  and  law 
enforcement  agencies  in  and  around 
Los  Angeles. 

Commenting  on  the  system,  Lt.  Mike 
Grossman,  manager  of  the  Force 
Training/Laser  Village  Training  Facility 
in  Los  Angeles,  says,  “It  doesn’t  get 
any  better.  It’s  really  a  great  opportuni¬ 
ty  to  be  able  to  participate  in  a  pro¬ 
gram  where  so  many  different  agen¬ 
cies  are  working  to  make  this  happen, 
and  be  able  to  provide  state-of-the-art 
training  for  Southern  California  -  for 
military,  federal,  state,  and  local  law 
enforcement.  I  think  the  sharing  of 
knowledge  and  expertise,  and  the 
joint  venture  doesn’t  get  any  better.  We 
appreciate  the  opportunity  to  be  the 
host  for  this  kind  of  operation.” 

A  second  system  will  be  available  for 
demonstration  at  relevant  trade  shows 
and  for  possible  temporary  installa¬ 
tions  at  select  user  agencies. 

Product  Concept  Evolves 

Cost  reduction  is  not  the  only  advan¬ 
tage  of  this  dual-use  effort.  Since  the 
cooperative  agreement  between  the 
Navy  and  its  industry  partner  was 
signed  in  February,  1996,  the  concept 
of  the  product  has  evolved. 

The  concept  for  the  commercially  pro- 
duced  system  now  incorporates 
marksmanship,  use-of-force  decision 
making,  special  weapons  and  tactics, 
and  advanced  military  weapon  team 


training  into  a  user-friendly,  easily 
upgraded  modular  system  design.2 

Other  Opportunities 

Other  opportunities  exist  for  collabo¬ 
ration  between  the  Department  of 
Defense  and  the  entertainment  indus¬ 
try.  Mechanisms  are  available  that 
encourage  the  government’s  collabora¬ 
tion  with  industry  to  conduct  joint 
research  and  development  (R&D). 
Under  this  framework,  the  govern¬ 
ment  gains  the  right  to  use  the 
research  results  for  government  pur¬ 
poses;  the  company  holds  all  commer¬ 
cial  rights.  Both  partners  share  the 
costs  of  conducting  the  research. 

Products  such  as  games  and  location- 
based  entertainment,  as  well  as  the 
underlying  technology  used  to  create 
entertainment  products  are  targets  of 
opportunity. 

Why  would  the  Navy  consider  part¬ 
ners  with  such  widely  diverse  motiva¬ 
tions  and  objectives?  Both  actively 
draw  from  modeling  and  simulation 
technologies,  to  produce  products. 

The  Navy  uses  commercial  games  in 
training  programs,  on  a  limited  basis. 
The  games  are  used  as  a  “backdrop”  to 
stimulate  behavior  —  such  as  coordi¬ 
nation  and  communication  between 
pilots  and  crew. 

Consider  the  sailor  or  student  of  today 
Many  have  hands-on  experience  — 
and  expertise  —  with  PC-based  learn¬ 
ing.  The  Navy  has  found  that  comput¬ 
er-based  games  provide  an  effective, 
low-cost  way  to  simulate  flying  and 
other  task  experiences.  The  applica¬ 
tions  must  be  appropriate  —  those  that 
do  not  require  expensive  hardware/ 
software  to  create  highly  accurate,  real¬ 
time  situations. 

The  joint  R&D  does  not  have  to  result 
in  a  product.  It  can  be  directed  at  the 
underlying  technology.  The  agree¬ 
ments  that  promote  this  collaboration 
are  not  covered  by  the  Federal  Acquisi¬ 
tion  Regulations,  which  apply  to  gov¬ 
ernment  contracts.  They  can  also  be 
exempted  from  the  Freedom  of  Infor- 
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mation  Act.  To  attract  these  commer¬ 
cial  partners,  the  government  recog¬ 
nizes  that  intellectual  property  must 
be  protected. 

Market  Dynamics 

These  types  of  agreements  help  move 
the  technology  out  of  the  laboratory 
and  into  the  marketplace.  The  technol¬ 
ogy  becomes  available  to  civilian  users, 
allowing  the  military  to  buy  resultant 
commercial  off-the-shelf  products. 

Invaluable  benefits  from  these  market 
dynamics  emerge,  as  a  broader  cus¬ 
tomer  base  lowers  the  per-unit  cost. 
The  military  is  getting  the  commercial 
price  to  acquire  a  system,  not  “cost- 
plus.”  Civilian  users  gain  the  benefits 
of  more  advanced  technology  (typical¬ 
ly,  in  the  area  of  learning  technology, 
where  the  Department  of  Defense  has 
the  lead).  We  will  see  more  of  this 
technology  moving  into  workforce 
development  and  K-12  education. 

The  rapid  pace  of  change  to  Depart¬ 
ment  of  Defense  acquisition  policy 
means  that  an  activity’s  internal  acqui¬ 
sition  policy  and  procedure  directives 
require  continuous  updating.  As  a 
result,  NAWCTSD  developed  the 
NAWCTSD  Acquisition  Guide ,  an  elec¬ 
tronic  acquisition  guide,  considered  to 
be  a  faster  method  of  communicating 
new  policy  to  NAWCTSD’s  own  acqui¬ 
sition  managers.3  First  introduced  in 
March  via  the  NAWCTSD  Website,  the 
guide  includes  an  Acquisition 
Roadmap,  which  is  a  tailored  represen¬ 
tation  of  the  Department  of  Defense 
acquisition  process,  as  revised. 

Endnotes 

1.  Clinton,  President  William  J.,  and 
Vice  President  Albert  Gore,  Jr.,  “Tech¬ 
nology  for  America’s  Economic 
Growth,  A  New  Direction  to  Building 
Economic  Strength”  (The  White 
House,  Feb.  22,  1993,  p.  14). 

2.  For  more  on  WTET,  visit 
http://www.ntsc.navy.mil/wtet/wtet. 
htm  at  NAWCTSD’s  Website. 

3.  To  view  or  access  the  NAWCTSD 
Acquisition  Guide ,  visit  http://www. 
ntsc.navy.mil/acqguide/acqguide. 
htm  at  NAWCTSD’s  Website. 


Cost  Analysis  Strategy  Assessment 
Model  (CASA)  Comes  of  Age 


It  Col.  Carl  Gardner,  U.S.Amty 


Wl 

v;  ■  ...  . -yy  y.  :  -  -  ...  :y-yyyyy  yyyyy  ,;y  •  yy \;;y.y  y,y  ~. yy  .y  y.  u- •.  y:y_yy; 

'  _ _  _ i-i _ C-J  J..  T _ CO. _ _  inn*  • 


flm  iifnr  Ane'At'C.c-  A A oT-ii &A} in  fVi o  toWr*  OA Q A’c  rnnfirmr'a H An  in rln d . 


'  ■  V  •  .  hy  1 . 

;  .Version  4,0  brings  the  ease  of  Windows™  to  its  users  and  allows  export  of. 

I  breakdown  structure  format)  allows  easy  data  entry.  The  flexibility  to  per-  . 

form  “What  if”  drills  is  increased  bv  the  addition  of  the, capability  to  vary 


,  1,  Manarv  Joel  M.,  “DSMC’s  CASA  Model  Still  Going  Strong,”  Program  . 

Mavaver  Magazine  Tanuarv-Februarv  1996:  - 


y  :  ...  '  :  •  m 

'  .  -  -  *  .  . 

l  nmcifric  XytnnnrroTOpnt  -T 


CASA  CAPABILITIES 


Cost  Estimating  1 1  C-  Analyses*  jg  § 


Warranty 


Spares  R^ource  Projections  Risk  and  Uncertainty-  h 

.Dr^innino  <e.g.,  Manpower,  ■  Ana[yses 


Analyses  "  -Provisioning  su^rt  Equip^nt) 


Cost  Driver  |  Reliability  Growth  •  '  Operational  '  I  Aut^al^Sensitivitv 
Sensitivity  Analysis  j  Analyses  Availability  ^alysis  ^ 


|  Operation  and 


- 


PM  :  SEPTEMBER-OCTOBER  1997  107 


MODELING  AND  SIMULATION 


National  Simulation  Superstructure 

Disney  Doesn't  Have  a  Monopoly 
on  the  World  of  "Make  Believe" 

KATHLEEN  M.  CLAYTON 


The  imagination,  creativity,  and 
technically  sophisticated  world 
of  “make  believe”  for  which  this 
city  is  famous  does  not  begin 
and  end  with  the  creations  of 
Walt  Disney.  The  Department  of 
Defense  (DoD)  is  the  fortunate  benefi¬ 
ciary  of  a  concentration  of  modeling 
and  simulation  (M&S)  expertise  -  a 
national  simulation  superstructure, 
also  located  in  Orlando.  This  collec¬ 
tion  of  collocated  defense  agencies, 
proven  M&S  companies,  academic 
institutions  with  M&S  curriculum, 
and  state  and  local  governments  is 
committed  to  the  enhancement  and 
use  of  this  leveraging  tool  as  a  vital 
national  resource. 

Recognizing  the  advantages  in  leverag¬ 
ing  this  array  of  talent  to  provide  the 
best  possible  products  to  the  Army,  the 
U.S.  Army  Simulation,  Training,  and 


Instrumentation  Command  (STRI- 
COM)  recently  led  the  formalization 
of  the  concepts  inherent  in  the  success 
of  this  unique  M&S  community.  Nam¬ 
ing  this  simulation  superstructure 
Team  Orlando ,  STRICOM  and  six 
other  key  players  (the  Naval  Air  War¬ 
fare  Center  Training  Systems  Division 
[NAWCTSD],  the  Joint  Simulation  Sys¬ 
tems  Office  [JSIMS],  the  Marine  Corps 
Program  Office,  the  Air  Force  Agency 
for  Modeling  and  Simulation  [AFAMS], 
the  Institute  for  Simulation  and  Train¬ 
ing  [1ST],  and  the  Training  and  Simula¬ 
tion  Technology  Consortium  [TSTC]) 
immediately  signed  on  as  Charter 
Members.  The  Team  Orlando  Charter 
outlines  the  synergy  of  the  group  as 
they  recommit  themselves  to  “work 
together  to  share  information  and 
leverage  programs  and  technology  in 
the  best  interest  of  the  Department  of 
Defense  and  the  American  taxpayer.” 


Origins 

Dating  from  the  1950s  two  Services, 
the  U.S.  Army  and  the  U.S.  Navy,  built 
and  enjoyed  a  special  relationship  in 
the  M&S  community.  The  informal 
concepts  behind  this  relationship  are 
the  basis  of  Team  Orlando.  The  success 
of  this  formula  provides  a  history  of 
mutual  benefits  for  both  Services. 
Today  STRICOM  and  the  Naval  Air 
Warfare  Center  Training  Systems  Divi¬ 
sion  (NAWCTSD)  benefit  greatly  from 
a  matured  inter-Service  relationship 
affording  each  entity  the  full  benefits 
of  leveraged  resources,  manpower,  and 

IST/UGF  Dy^micTerrajn  Area  of  Visual  — 

■  .Systems  Laboratory  :  v :  -  : 


Clayton  is  a  Project  Director  at  the  U.S.  Army  Simulation,  Training,  and  Instrumentation  Command,  Orlando,  Florida  She  holds  a  BA  from  Arizona  State  Univer¬ 
sity  and  an  M.A.  from  Georgia  State  University. 
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a  Transfer  Center  Classroom. 

technological  expertise,  creating  a 
strong  foundation  of  success  and  the 
springboard  for  joint  projects,  such 
as  JSIMS.  The  Marine  Corps  Program 
Office,  another  tenant  of  NAWCTSD, 
works  as  the  principal  representative 
for  ground  and  air  M&S  programs 
impacting  the  U.S.  Marine  Corps. 
The  U.S.  Air  Force,  also  seeing  the 
benefit  of  leveraging  the  resources  in 
the  area,  established  the  Air  Force 
Agency  for  Modeling  and  Simulation 
(AFAMS)  in  Central  Florida  and  plans 
to  grow  this  organization  in  the  next 
few  years. 

The  State  of  Florida  recognized  this 
growing  industry  by  establishing  the 
Institute  for  Simulation  and  Training 
at  The  University  of  Central  Florida 
(IST/UCF)  in  1982.  The  1ST  provides 
a  common  source  of  academic  studies 
and  research  in  support  of  the  M&S 
community 

In  1985,  the  Governor  and  Cabinet  of 
the  State  of  Florida  issued  a  resolution 
recognizing  the  “Center  of  Excellence 
for  Simulation.”  Today,  Enterprise 
Florida,  as  the  state’s  economic  devel¬ 
opment  unit,  recognizes  the  significant 
contribution  of  STRICOM,  NAW¬ 
CTSD,  the  Marine  Corps  Program 
Office,  and  the  growing  AFAMS  as  the 
mainstays  of  the  Center  of  Excellence. 

In  1993,  as  further  testament  to  the 
capability  of  this  unique  M&S  com¬ 
munity,  the  TSTC  was  established 
under  the  White  House  Technology 
Reinvestment  Project.  The  TSTC  was 
chartered  to  be  a  non-profit,  one-stop 
source  for  all  commercial  applications 
of  these  sophisticated  military  and 
space  M&S  technologies. 

feam  Orlando  in  Action  — 

DoD  Membership 

Over  1700  M&S  professionals,  repre¬ 
senting  the  four  primary  uniformed 
services  of  the  United  States,  comprise 
the  government  contingent  of  Team 
Orlando .  The  relationships  and  inte¬ 
gration  between  and  among  these 


dedicated  professionals  is  where  the 
benefits  of  leveraging  begin.  Coopera¬ 
tive  efforts  between  these  government 
professionals,  industry,  and  academia 
work  to  realize  the  benefits  for  the 
warfighter  and  the  taxpayer. 

One  of  the  best  examples  of  the  Team 
Orlando  concepts  in  action  is  exempli¬ 
fied  by  the  symbiotic  relationship 
between  STRICOM,  NAWCTSD,  and 
the  Marine  Corps  Program  Office.  The 
organizations  share  facilities,  with 
STRICOM  and  the  Marine  Corps  Pro¬ 
gram  Office  as  tenants.  STRICOM  and 
the  Marine  Corps  Program  Office  buy 
various  types  of  base  operations/infra¬ 
structure  support  services,  and  work 
years  of  contracting  and  engineering 
talent  from  NAWCTSD.  This  arrange¬ 
ment  benefits  all  three  organizations 
by  sharing  expertise,  techniques,  and 
methodologies,  further  enhancing 
integration  and  synergy  among  the 
Services. 

The  growing  U.S.  Air  Force  presence 
in  this  national  simulation  superstruc¬ 
ture  promises  an  additional  potential 
to  develop  systems  that  combine  the 
best  of  the  Services.  The  mission  of 
AFAMS  is  to  implement  DoD,  Joint, 
and  Air  Force  M&S  Policy/Standards 
and  provide  Service-level  M&S  sup¬ 
port  to  Joint,  Combined,  and  Air  Force 
Activities.  This  office  also  supports  Air 
Force  and  Joint  Wargaming  Exercises, 
and  supports  other  major  joint  M&S 


initiatives,  such  as  the  Joint  Modeling 
and  Simulation  System  (JMASS). 
Clearly,  the  synergy  provided  by  Team 
Orlando  will  be  vital  as  AFAMS,  STRI¬ 
COM,  and  NAWCTSD  support  and 
manage  the  development  of  joint  syn¬ 
thetic  battlespace  and  advanced  dis¬ 
tributed  simulations  of  their  respective 
Service  customers. 

Additionally,  the  JSIMS  mission  pro¬ 
vides  Commanders  in  Chief  and  the 
Services  with  next  generation  training, 
mission  planning,  and  mission 
rehearsal  capabilities.  The  JSIMS  Joint 
Program  Office  values  the  benefits  of 
the  external  joint  community  and  the 
synergy  created  under  Team  Orlando. 

The  DoD  members  of  Team  Orlando 
also  include  representatives  from 
Army  Research  Institute,  Army 
Research  Laboratory,  Army  National 
Guard,  JSIMS  Maritime,  U.S.  Naval 
Reserve,  U.S.  Marine  Corps  Reserve, 
Air  Force  Materiel  Command  Operat¬ 
ing  Location,  and  representatives  from 
our  NATO  allies  in  Germany  and  the 
United  Kingdom.  Each  of  these  orga¬ 
nizations  capitalize  on  the  opportunity 
to  use  M&S  solutions  to  their  full 
advantage  in  fulfilling  their  individual 
missions. 

Other  Government  Agency 
Membership. 

In  the  spirit  of  Team  Orlando ,  STRI¬ 
COM  and  National  Aeronautics  and 
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Space  Administration  (NASA)  recently 
signed  an  Interagency  Agreement  for 
Technology  Cooperation.  This  agree¬ 
ment  serves  as  a  foundation  for  a  more 
lasting  technology  transfer  program 
and  continuous  business  relationship. 
By  cultivating  this  business  relation¬ 
ship,  STRICOM  and  NASA  promote 
appropriate  simulation  and  related 
technology  for  national  Service,  space, 
and  other  applications.  Delineated  in 
the  agreement  is  the  sharing  of  infor¬ 
mation,  technologies,  methodologies, 
consultation,  and  other  services;  and 
working  toward  further  collaboration 
of  efforts  involving  simulations,  simu¬ 
lators,  and  instrumentation  technolo¬ 
gy  and  methodologies.  Also  included 
are  joint  research  or  specific  projects, 
whenever  possible. 

The  Florida  High-Technology  Corridor 
Council  and  Enterprise  Florida  are 
working  together  with  STRICOM, 
NAWCTSD,  and  approximately  150 
companies  involved  in  modeling,  sim¬ 
ulation,  and  training  activities  in  Cen¬ 
tral  Florida  to  provide  for  continued 
growth  and  recognition  of  the  impor¬ 
tance  of  this  national  asset.  This 
national  simulation  superstructure, 
guided  by  the  concepts  of  Team  Orlan¬ 
do,  is  growing  and  attracting  new  sim¬ 
ulation  interests  every  year. 

Academic  Membership 

Located  adjacent  to  STRICOM  and 
NAWCTSD,  the  University  of  Central 


Florida  (UCF),  Institute  for  Simulation 
and  Training  (1ST)  works  to  fulfill  its 
mission  as  a  resource  and  focal  point 
for  simulation  and  training  technolo¬ 
gies.  The  synergism  generated  through 
Team  Orlando  enables  1ST  to  play  a 
key  role  in  advancing  the  art  of  simula¬ 
tion  and  training  technologies  and  the 
transfer  of  those  technologies  to  the 
civilian  sector.  These  efforts  enhance 
our  society  and  get  the  most  return 
from  research  dollars. 


As  a  charter  member  of  Team  Orlando , 
UCF  takes  an  aggressive  interest  in 
simulation  and  training.  The  first  uni¬ 
versity  in  the  nation  offering  a  masters 
degree  in  simulation,  UCF  is  currently 
developing  a  Ph.D.  program.  DoD 
employees  are  encouraged  to  take  full 
advantage  of  these  resources. 

Industry  Players 

DoD  Team  Orlando  members  have  sig¬ 
nificant  involvement  with  many  of  the 
150  commercial  industry  partners 
located  here  in  Central  Florida.  Many 
serve  as  a  contractor  or  subcontractor 
on  crucial  DoD  programs,  supporting 
$1  billion  in  contracts  annually  As  we 
partner  with  local  industry,  Team 
Orlando  DoD  members  benefit  from 
the  unique  opportunity  to  take  advan¬ 
tage  not  only  of  federal  Acquisition 
Reform  initiatives,  but  also  reap  the 
rewards  provided  by  the  synergy  creat¬ 
ed  by  this  simulation  superstructure. 

Many  other  industry  players  are 
embarking  with  Team  Orlando  on  a 
journey  toward  dual-use  exploration. 
For  example,  STRICOM  and  the  Walt 
Disney  Company,  the  premier  expert 
on  the  use  of  M&S  in  the  entertain¬ 
ment  industry,  have  an  ongoing  rela¬ 
tionship  exploring  dual  use  and  tech¬ 
nology  transfer  opportunities. 

The  mission  of  TSTC  is  to  assist  pri¬ 
vate/commercial  industry  to  acquire 
simulation  and  training  technologies 
and  capabilities  previously  available 
only  to  the  U.S.  military  and  space 
effort.  TSTC  membership  includes 


:  )I^UCF  Virtual  Environment  Testbed. 
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Performance  Tech  Lab 


-Distributed  Training  :  ;; 
-Performance  Technology 
-Technology  Integration  : 
-Total  Systems  Analysis 


Simulation  Lab 


-3D  Virtual  Sandtable 
-Synthetic  Environment 
-;Recohfigurable. 
Simulations 


Lair 


f 

■jr 


-After  Action  Review 
./'  •  -Standards  and  Protocols 
-Linking  of  Simulations 
f V  -- -HLA  Testbed  ■’ 


Computer  Generated 
Forces  Lab 


-Intelligent  Simulated 
Forces 

-Integrated  Eagle/BDS-D 
-SAF  Dismounted  Infantry 


Information 

Technology 


-WWW  Integration 
-Information  Integration 
-Technology  Forecasting 
-Information  Distributed 


Visual  Systems  Lab 


-Dynamic  Terrain 
.  -Hydrology 
-Virtual  Reality  : 


STRICOM,  NAWCTSD,  AFAMS, 
NASA,  1ST,  and  over  20  private  compa¬ 
nies.  One  project  under  exploration  by 
the  TSTC  is  a  regional  transportation 
planning  system,  for  which  the  tech¬ 
nology  is  applicable  to  the  national 
defense  and  its  readiness,  as  well  as 
federal,  state,  regional,  and  local  trans¬ 
portation  planners.  TSTC  is  also 
exploring  other  simulation  and  train¬ 
ing  projects  with  the  American  Red 
Cross,  Universal  Studios,  Kennedy 
Space  Center,  and  the  Federal  Emer¬ 
gency  Management  Agency  (FEMA). 

The  work  of  TSTC  —  to  raise  aware¬ 
ness  and  facilitate  technology  transfer 
and  jobs  —  clearly  make  it  an  asset  to 
Team  Orlando  by  sustaining  the 
defense  industrial  base  upon  which  the 
Department  of  Defense  must  rely  for 
affordable,  state-of-the-art  M&S  tech¬ 
nology  development  and  applications. 

Recognized  Results 

As  a  member  of  this  national  simula¬ 
tion  superstructure  —  Team  Orlando  — 
success  is  multiplied  among  all  the 
members.  This  national  simulation 
superstructure  provides  DoD  with  an 
enormous,  technologically  advanced 
support  unit.  Our  contractors  provide 
us  with  better  products  through  reten¬ 
tion  of  highly  skilled  jobs,  advance¬ 
ments  in  the  M&S  industry,  and  acad¬ 
emic  support  for  the  technical 
educational  needs  required  by  the 
simulation  industry. 

The  value  of  this  national  simulation 
superstructure,  led  by  Team  Orlando , 
can  be  seen  daily  in  many  areas: 

•  Growing  Number  of  High-Technol- 
°gy  Jobs 

•  Reduced  Cost  for  DoD  End  Items 

♦Shorter  Time  from  Technology 

Development  to  End  User  Applica¬ 
tions 

•Number  of  Strong  Bidders  for 
DoD  Contracts 

•Number  of  Cooperative  Research 
and  Development  Agreements 
(CRADA) 

•  Grants  And  Cooperative  Agree¬ 
ments  Integrating  Collaboration  in 
Research  and  Development 


Team  Orlando  members  plan  to  con¬ 
tinue  to  expand  these  contacts  and 
agreements,  leveraging  and  enhancing 
the  innovations  being  developed  by 
and  between  the  growing  membership 
of  the  team  for  the  overall  long-term 
benefit  of  all. 

This  growing  force  of  government, 
industry,  and  academia  M&S  experts 
share  a  common  vision  for  the  future 
of  M&S  and  its  recognized  develop¬ 
ment.  The  continued  success  of  Team 
Orlando  is  vital  to  the  shared  goal 
of  cost  reduction  by  leveraging  of 
DoD  M&S  dollars.  By  working  to 
preserve  and  advance  the  industrial 
base,  increase  the  willingness  of  indus¬ 
try  to  invest  their  R&D  efforts  in 
M&S,  foster  innovative  applications 
of  the  latest  technology,  and  lev¬ 


erage  the  numerous  M&S  projects  in 
other  areas,  the  Department  of 
Defense,  through  the  efforts  of  Team 
Orlando ,  can  maintain  its  technologi¬ 
cal  superiority  at  a  reduced  cost  into 
the  future. 

The  Team  Orlando  model  is  a  success 
story.  If  fostered,  it  will  assure  the 
nation  a  network  of  highly  qualified 
companies  ready  and  able  to  develop 
superior  M&S  technologies  and  prod¬ 
ucts  for  the  Department  of  Defense.  As 
written  in  the  STRICOM  crest,  “All  But 
War  Is  Simulation.”  These  state-of-the- 
art,  “make  believe”  M&S  solutions  will 
provide  the  warfighters  of  tomorrow, 
operating  under  the  concepts  of  Joint 
Vision  2010,  with  capabilities  to  ensure 
the  success  of  U.S.  Forces  into  the 
next  millennium. 
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Integrated  Ship  Defense  llodeling  and 
Simulation  Pilot  Program 

If  PNs  Bring  N&S  Into  Focus  DoD-Wide,  They’ll  See 
a  Real  Return  on  Investment 

LORRAINE  SHEA  •  MICHAEL  POBAT 


Can  modeling  and  simulation 
(M&S)  truly  be  a  highway  for 
the  program  manager  to  navi¬ 
gate  the  road  to  project  success 
over  the  life  cycle?  Currently, 
the  acquisition  community  is  embrac¬ 
ing  Simulation  Based  Acquisition 
(SBA)  initiatives,  but  where  is  the  evi¬ 
dence  that  there  is  a  payoff  here? 
Where  is  the  real  value-added? 

Traditionally,  program  managers  navi¬ 
gate  the  life-cycle  process  in  different 
ways  using  a  variety  of  available  tools, 
including  M&S.  So  what  is  new  here? 
What  is  this  M&S  revolution  all 
about? 

As  a  system  grows  throughout  the 
engineering  and  development  phase, 
SBA  -  when  used  by  the  engineers 
who  are  designing  the  system  and  the 
platform  it  will  ride  on;  analysts  per¬ 
forming  trade  studies  and  investment 
analyses;  and  testers  responsible  for 
certifying  the  design  meets  specifica¬ 
tions  —  allows  a  conceptual  model  to 
grow  in  functionality  and  increasing 
specification.  The  end  result  is  a  well- 
understood,  credible  representation  of 
that  system,  capable  of  augmenting 
developmental  and  operational  testing. 
This  same  model  can  then  be  passed 
to  the  in-service  and  training  commu¬ 


nity  for  use  during  deployment  and 
Pre-Planned  Product  Improvements. 
Although  the  level  of  abstraction  of  the 
basic  model  may  change  from  applica¬ 
tion,  a  pedigree  is  established  based 
on  a  common  system  representation 
that  becomes  the  standard  for  any 
application.  Hence,  an  adaptive  life- 
cycle  tool  evolves  for  the  program 
manager. 


Program  managers  then,  gain  the  ben¬ 
efit  of  a  readily  available  engineering 
model  of  the  system  that  assists  in  the 
design  and  development  process,  and 
is  reusable  and  interpretable,  not  only 
with  other  elements  of  the  overall  sys¬ 
tem,  but  with  the  entire  technical  and 
operational  community.  Regardless  of 
the  design  agent,  laboratory,  field 
activity,  or  Fleet  installation,  the  foun- 


Sheo  is  the  Deputy  Director  for  Modeling  and  Simulation,  Program  Executive  Office,  Theater  Air  Defense  (PEOfTADJ),  Systems  Engineering  Division,  Arlington,  Va.  A 
Department  of  Navy  employee  since  1989,  Shea  previously  worked  at  Headquarters,  Naval  Sea  Systems  Command  (NAVSEA),  where  she  supported  the  test  and 
evaluation  community  as  a  weapons  systems  engineer,  providing  on-site  support  to  the  Fleet,  at  sea  and  at  shore-based  installations. 

Pobat  is  the  Integrated  Ship  Defense  Program  Manager,  Weapon  Systems  Division,  for  Litton/PRC,  Inc.,  located  in  Arlington,  Va.  For  the  past  five  years,  his  work 
with  the  Navy's  PEOfTAD)  has  included  Ship  Self  Defense  investment  strategy  development  and  modeling  and  simulation.  Pobat  previously  spent  10  years  in  the 
Navy  as  an  Electronic  Warfare  Technician,  with  tours  aboard  the  U.S.S.  Biddle  (CG-34),  U.SS  Austin  (LPD-4),  and  U.S.S.  Sterett  (CG-31).  Homeported  in  Subic  Bay 
Philippines  for  two  years,  he  also  completed  tours  at  various  stateside  shore  stations 
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dation  exists  for  the  operation  of  and 
interaction  between  the  system  mod¬ 
els.  When  you  begin  to  think  of  the 
flexibility  SBA  allows  and  the  time  it 
can  save,  the  payoffs  become  evident. 
Ultimately,  SBA  enables  us  to  develop, 
field,  and  support  the  best  products  to 
the  operational  community  in  a  more 
cost-effective  way 

Current  technology  can  support  this 
revolution.  Now  is  the  time  for  the 
acquisition  community  to  be  creative 
and  integrate  this  technology  with 
sound  engineering  practices. 

Selection  of  the  Pilot  Program 

In  1995,  the  Program  Executive  Office 
(Theater  Air  Defense)  (PEO[TAD]) 
Technology  Directorate  proposed  a  set 
of  Advanced  Distributed  Simulation 
(ADS)  Pilot  Programs  that  was,  in 
part,  prompted  by  the  1994  Naval 
Research  Advisory  Committee 
(NRAC)  study  The  NRAC  study 
endorsed  the  use  of  ADS  in  support  of 
the  acquisition  process  and  stated  that 
“DoN  [Department  of  Navy  ]  acquisi¬ 
tion  that  would  provide  good  candi¬ 
dates  for  Distributed  Simulation  Based 
Acquisition  (DSBA)  are  mine  counter¬ 
measures,  sea-based  Theater  Ballistic 
Missile  Defense  (TBMD),  and  Ship 
Self  Defense.”  Based  on  these  differing 
mission  areas,  the  PEO(TAD)  pro¬ 
posed  three  specific  programs  as 
potential  pilot  programs:  Integrated 
Ship  Defense  (ISD),  TBMD,  and  Over¬ 
land  Cruise  Missile  Defense.  Ultimate¬ 
ly,  the  Navy  selected  the  ISD  Pilot 
because  it  represented  the  most 
mature  and  current  Fleet  sensor/ 
weapon  system. 

In  May  1996,  the  Office  of  Naval 
Research  tasked  PEO(TAD)  to  further 
develop  the  ISD  Pilot  Program  con¬ 
cept  and  provide  a  detailed  program 
plan.  A  team  consisting  of  representa¬ 
tives  from  PEO(TAD),  Naval  Surface 
Warfare  Center  Dahlgren,  Naval 
Research  Laboratory,  Johns  Hopkins 
University/ Applied  Physics  Laboratory, 
the  Mitre  Corporation,  and  PRC  Inc., 
provided  the  necessary  subject  matter 
experts  for  the  task.  Completed  in 
September  1996,  the  ISD  Pilot  Pro¬ 


gram  Plan  provides  the  detailed  tech¬ 
nical  and  programmatic  aspects.  To 
generate  support  and  solicit  feedback 
on  the  proposed  ISD  Pilot  Program, 
the  team  conducted  a  series  of  key 
briefings  to  solicit  feedback,  guidance, 
and  support  from  key  DoD/DoN 
senior  civilian  and  military  personnel. 
As  a  result,  they  gathered  enough 
information  from  the  following  offices 
to  transform  the  Pilot  Program  Plan 
into  an  executable  program: 


•  Office  of  the  Secretary  of  Defense 
(OSD)  Director  of  Research  and 
Engineering 

•  OSD  Director  of  Test  Systems  Engi¬ 
neering  and  Evaluation 

•  Assistant  Secretary  of  the  Navy  for 
Research,  Development,  and  Acqui¬ 
sition  (C4I) 

•  Defense  Modeling  and  Simulation 
Office 

•  Chief  of  Naval  Operations 

•  Director  of  Navy  Test  and  Evalua¬ 
tion  and  Technology  Requirements 
(N091) 

•  Navy  Modeling  and  Simulation 
Office  (N6M) 

ISD  Program  Description 

The  Chief  of  Naval  Operations 
approved  a  plan  for  development  of  a 
Quick  Reaction  Combat  Capability 
(QRCC)  to  improve  defenses  against 
anti-ship  cruise  missiles  for  non-AEGIS 
ships,  and  to  assure  greater  survivabili¬ 
ty  for  ships  operating  in  harm’s  way. 

To  effectively  defend  against  an 
increasingly  stressing  cruise  missile 
threat,  the  operator  requires  an  auto¬ 
mated  detect-through-engage  capabili¬ 
ty  with  reduced  reaction  time.  The 
operator  then  has  the  capability  to 
associate  and  correlate  multi-sensor 
data  to  provide  a  sensor-fused  com¬ 
posite  track  that  assures  a  high  level  of 
certainty  in  target  identification  and 
classification.  Use  of  flexible  doctrine 
that  supports  layered  defense  engage¬ 
ments  provides  the  operator  automat¬ 
ed  control  of  the  system  functions  and 
actions.  Once  the  system  presents  and 
displays  the  information  such  that  the 
operators  have  an  accurate,  precise, 
and  comprehensive  picture  of  the  tac¬ 
tical  situation,  the  operator  can  then 
override,  abort,  or  alter  doctrine  as 
necessary  Ultimately,  the  intent  is  to 
provide  a  fully  automated  ISD  capabil¬ 
ity. 

The  ISD  combat  system  provides  auto¬ 
mated  detection-control-engagement 
by  integrating  existing  stand-alone 
weapons  and  sensors  via  the  Ship  Self 
Defense  System  (SSDS)  MK-1.  Such 
integration  involves  a  series  of  auto¬ 
mated  actions/reactions: 
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Figure  1  The  "Sneaker  Met" 
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•  Manual  Interaction 
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•  Existing  sensors  detect  targets  and 
provide  track  data  to  distributed 
track  file  processors  via  a  Local  Area 
Network  (LAN). 

•  Each  track  file  processor  correlates 
and  associates  track  data  for  use  by 
the  SSDS  in  Sensor  Integration  and 
Control  processors,  which  assign 
and  manage  common  track  file 
numbers. 

•  The  Local  Command  and  Control 
processor  determines  target  identifi¬ 
cation,  classification,  and  appropri¬ 
ate  action. 

•  The  Weapon  Integration  and  Con¬ 
trol  processors  manage  scheduling. 
Providing  a  layered  defense  that 
ensures  the  best  employment  of 
hardkill  and  electronic  warfare 
(HK/EW)  segments,  these  proces¬ 
sors  automatically  determine  the 
weapon(s)  mix  required  to  defeat 
the  threat. 

Current  ISD  H&S  Capability 

The  ISD  Pilot  Program  includes  a  fed¬ 
eration  of  interactive  hi-fidelity  models 

built  upon  and  from  the  existing  fami¬ 


ly  of  credible,  authoritative  (although 
primarily  stand-alone)  ISD  M&S.  Sep¬ 
arate  program  offices  originally  devel¬ 
oped  these  legacy  M&S  to  aid  engi¬ 
neers  in  design,  development,  test  and 
evaluation  (performance  prediction), 
and  planning.  With  the  formulation  of 
the  ISD  program  office  and  a  focus  on 
the  integrated  combat  system  opera¬ 
tion,  a  need  surfaced  to  integrate  the 
models  as  well.  A  team  of  subject  mat¬ 
ter  experts  from  various  laboratories 
and  government  facilities  manually 
integrate  the  models  and  conduct 
combat-system-level  analysis  such  as 
Program  Objectives  Memorandum 
investment  strategies;  cost  and  opera¬ 
tional  effectiveness  analyses  (COEA) 
or  Assessment  of  Alternatives  (AOA); 
and  selected  ship-class  performance 
capability  studies.  This  manual  inte¬ 
gration  is  known  as  “the  Sneaker  Net” 
(Figure  1). 

The  sneaker  net  is  literally  the  human- 
in-the-loop,  which  hand-carries  the 
results  of  one  model  to  the  operator  of 
the  next.  This  process  is  labor-  and 


time-intensive  and  does  not  capture 
many  benefits  inherent  in  the  SSDS 
and  QRCC.  The  current  M&S  capa¬ 
bility,  although  sufficient  for  the  appli¬ 
cations  mentioned,  does  not  provide 
the  level  of  fidelity  and  operational 
realism  required  for  the  SBA  environ¬ 
ment  (i.e.,  common  battlespace, 
reactive  threat,  jamming,  realistic 
equipment  availability,  hi-fidelity  mod¬ 
eling  of  Electronic  War  fare/ Infrared 
(EW/IR),  Hardkill/Electronic  Warfare 
(HK/EW),  and  common  standardized 
databases  that  are  usable  by  all  inter¬ 
active  simulations). 

The  demand  for  more  operationally 
realistic  M&S  capability  (e.g.,  threats, 
system  availability,  environment,  etc.), 
a  deeper  understanding  of  HK/EW 
layered  defense,  and  a  means  of  inte¬ 
grating  geographically  distributed 
engineering  models  and  subject  mat¬ 
ter  experts,  highlight  the  need  for  a 
new  approach  to  M&S. 

ISD  Technical  Issues 

The  ISD  Pilot  Program  addresses  the 
shortfalls  of  the  existing  M&S  capa¬ 
bilities  (i.e.,  the  Sneaker  Net). 
Improvements  incorporate  reactive 
threats  and  operational  environments 
to  increase  the  realism  and  credibility 
of  the  results.  As  a  first  step,  it  builds 
upon  an  established  set  of  existing 
engineering-level  models  with  known 
capabilities,  by  linking  them  together 
via  a  High  Level  Architecture-compli¬ 
ant  Run-Time  Infrastructure  (RTI). 
Ultimately,  the  ISD  Pilot  Program 
must  address  the  following  technical 
issues: 

•  Evaluate  and  quantify  weapons  and 
threat  interaction  (performance) 
with  the  environment  (reactive 
threat,  dual  mode  RF/IR). 

•  Evaluate  and  quantify  weapons 
interaction  (performance)  with  the 
threat. 

•  Evaluate  and  quantify  sensors’  inter¬ 
action  with  threat  and  environment. 

•  Evaluate  and  quantify  HK  envelopes 
for  probability  of  kill. 

•  Evaluate  and  quantify  HK  and  EW 
weapons  interactions  and  effective¬ 
ness. 
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•  Generate  accurate  and  repeatable 
system  analysis  data  for  ISD  verifica¬ 
tion  and  isolation  of  problems. 

•  Evaluate  and  quantify  system  effec¬ 
tiveness  using  performance  mea¬ 
sures. 

•  Create  a  common-usage,  controlled 
environment  for  demonstration  of 
system  modifications  and  standard¬ 
ization  of  threat,  environment,  and 
scenario  representations. 

Program  managers  must  address  and 
solve  these  technical  issues  through  a 
thorough  understanding  of  the  capa¬ 
bilities,  limitations,  and  interactions  of 
a  number  of  diverse  weapons  and  sen¬ 
sors  in  complex  land,  sea,  and  littoral 
environments.  To  evaluate  system  per¬ 
formance,  hi- fidelity,  physics-based 
engineering  simulations  must  reflect 
these  complex  system  interactions  as 
well  as  dynamic  environmental  effects. 
Consideration  of  these  interdependen¬ 
cies  between  sensors  and  weapons; 
weapons  and  threats;  and  between 
sensors,  weapons,  and  the  environ¬ 
ment,  dictates  a  departure  from  the 
traditional  isolated  system  and  subsys¬ 
tem  engineering  analyses  and  simula¬ 
tions. 


In  the  past,  program  managers  studied 
these  interdependencies  in  the  real 
world,  through  expensive  exercises 
and  testing.  Regrettably,  in  many  cases 
the  complexity  of  today’s  weapons  sys¬ 
tems  surpasses  the  affordability  of 
complete  testing  in  real-world  exercis¬ 
es.  The  simulations  proposed  for  the 
ISD  Pilot  Program  will  provide  the 
capability  to  conduct  a  large  part  of 
these  analyses  and  evaluations  without 
expending  costly  ship,  personnel,  and 
test  and  evaluation  resources,  and  lay 
the  groundwork  for  advancing  SBA  ini¬ 
tiatives. 

ISD  Pilot  Program  Overview 

The  goal  of  the  ISD  Pilot  Program  is  to 
develop  and  demonstrate  a  compre¬ 
hensive  M&S  capability  that  supports 
the  design  and  evaluation  of  compo¬ 
nents  and  systems,  which  further  sup¬ 
port  SBA  initiatives.  The  ISD  ADS  Pilot 
Program  will  be  conducted  over  a 
period  of  three  years.  Each  phase  will 
retain  its  own  set  of  objectives;  howev¬ 
er,  each  phase  will  build  on  the  capa¬ 
bilities  demonstrated  in  the  preceding 
phase.  Figure  2  shows  the  three  phases 
of  the  program  and  the  evolving  capa¬ 
bilities.  The  goal  is  to  increase  the  sim¬ 


ulation  set  and  proceed  toward  the 
eventual  implementation  of  the  super¬ 
set  of  simulations.  A  brief  description 
of  each  phase  follows. 

Phase I 

The  development  team  intended  that 
this  initial  phase  provide  a  benchmark¬ 
ing  opportunity  in  the  development  of 
ISD  Federation.  Accordingly,  the  sys¬ 
tem  designers,  modelers,  and  testers 
will  be  addressing  the  complex  issues 
inherent  to  test  and  evaluation.  Of  par¬ 
ticular  interest  is  the  ability  to  perform 
HK/EW  integrated  modeling  in  a  dis¬ 
tributed  environment  using  a  High 
Level  Architecture-compliant  RTI.  For 
this  reason,  the  approach  is  conserva¬ 
tive  and  is  tailored  to  achieve  the  great¬ 
est  capability  in  a  one-year  time  peri¬ 
od.  This  time  period  will  still  permit 
the  development  team  to  gain  the 
experience  needed  to  accomplish 
more  complex  configurations  in  sub¬ 
sequent  phases.  To  minimize  risks,  the 
simulations  will  be  developed  at  the 
developer’s  site.  The  integration,  how¬ 
ever,  will  be  accomplished  in  a  single 
laboratory,  with  the  simulations  inter¬ 
connected  via  RTI,  but  using  a  LAN. 
The  products  of  Phase  I  are  — 


Figure  2  Evolving  Capabilities 
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•  first-time,  hi-fidelity  detect-through- 
engage  simulation  capability; 

•  hi-fidelity,  integrated  HK/EW  assess¬ 
ment  capability; 

•  threat  reactive-common  to  all  com¬ 
bat  system  elements; 

•  contribution  to  Joint  Synthetic  Test 
and  Evaluation  batdespace; 

•  established  foundation  for  Phases  II 
and  III; 

•  PEO(TAD)  established  as  a  beta  test 
site  for  Defense  Modeling  and  Simu¬ 
lation  Office  RTI;  and 

•  verification  and  validation  of  federa¬ 
tion. 

Figure  3  depicts  the  architecture  for 
Phase  I  development. 

Phase  II 

The  intent  in  Phase  II  is  to  use  the 
experience  gained  in  Phase  I  to 
greatly  increase  the  capability  of  the 
federation  through  the  incorporation 
of  additional  federates.  This  com- 
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plexity  will  enable  a  close  examina¬ 
tion  of  sensor  integration  and  will 
permit  a  systematic  approach  to  the 
investigation  of  HK/EW  coordination. 
Models  involved  in  this  phase  will 
reside  at  the  developer’s  site  and  will 


be  interconnected,  through  the  RTI, 
via  a  geographically  distributed  net¬ 
work. 

Additional  reactive  threats  will  be 
added  in  this  phase.  The  intent  is  to 


Figure  4  Phase  II  Architecture 
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add  threats  whose  performance  can 
stress  the  capabilities  of  the  ISD  com¬ 
bat  system.  In  this  way  the  federation 
can  be  used  to  explore  reaction  times 
of  different  combat  system  configura¬ 
tions  to  stressing  situations.  This  will 
also  permit  an  evaluation  of  the  feder¬ 
ation  and  its  capability  to  simulate 
real-time  operation.  The  products  of 
Phase  II  are  - 

•  active  electronic  attack  assessment; 

•  realistic  representation  of  opera¬ 
tional  environment; 

•  geographically  distributed  simula¬ 
tion  using  Asynchronous  Transfer 
Mode/Sonet  Network; 

•  network  technology  that  provides 
feasibility  of  a  re-use  tool; 

•  verification  and  validation  of  federa¬ 
tion;  and 

•  additional  threat  families  represent¬ 
ed. 

Figure  4  depicts  the  architecture  for 
Phase  II  development. 

Phase  III 

The  intent  of  Phase  III  is  to  produce  a 
federation  that  provides  a  capability  to 
model  conceptual  systems  of  the  next 
generation  combat  system  -  Akcita. 
This  will  enable  the  federation  to  sup¬ 
port  SBA  initiatives  for  future  acquisi¬ 
tion  programs. 

To  provide  a  realistic  operational  envi¬ 
ronment,  this  phase  will  complete  the 
addition  of  propagation,  clutter,  and 
weather  models  to  achieve  a  dynamic 
multispectral  environment.  This  will 
enable  the  examination  of  both  Radio 
Frequency  (RF)  and  Infrared  (IR) 
threats  in  a  stressing  environment.  To 
provide  detection  of  these  dual-mode 
threats,  this  phase  also  adds  an  IR  sen¬ 
sor. 

The  Gateway  Federate  will  be  em¬ 
ployed  and  tested  in  this  phase, 
enabling  communication  and  interac¬ 
tion  between  two  federations  of  differ¬ 
ing  levels  of  fidelity  and  resolution. 
The  intent  is  to  link  the  ISD  Federa¬ 
tion  to  the  Joint  Countermine  Opera¬ 
tional  Simulation  (JCOS)  Federation 
to  simulate  a  multi-warfare  exercise. 
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This  would  permit  inter-federation 
communications  between  a  federa¬ 
tion  operating  with  engineering-level 
simulations  and  a  federation  operat¬ 
ing  at  an  engagement  simulation 
level  (be.,  lower  fidelity).  Phase  III 
products  include  — 

•  IR  sensor,  environment,  and  threat 
modeling; 

•  conceptual  ship  and  combat  sys¬ 
tem  models; 

•  advanced  threat  models  (full  com¬ 
plement  of  ISD  threat  representa¬ 
tive  models); 

•  advancement  of  SBA  initiatives 
through  multi-fidelity  simulation; 

•  inter-federation  linking  (Gateway 
Federate);  and 

•  verification  and  validation  of  feder¬ 
ation. 

Figure  5  depicts  the  architecture  for 
Phase  III  development. 

Value-Added  and  Support  to 
Acquisition  Program  Manager 

The  tools  resulting  from  completion 
of  the  Pilot  Program  have  the  poten¬ 
tial  to  enhance  the  system  acquisi¬ 
tion  process  by  adding  value  in  the 
following  areas: 

•  AOA.  The  federation  of  ISD  analyt¬ 
ical  models  can  be  used  to  deter¬ 
mine  operational  effectiveness 
against  specified  threats  as  part  of 
an  AOA  Study. 

•  Mission.  As  a  means  of  developing 
a  Requirements  Definition,  the 
simulations  provide  a  means  for 
quantitative  evaluation  of  mea¬ 
sures  of  effectiveness  and  perfor¬ 
mance  prior  to  verifying  system 
requirements. 

•  System  Engineering.  The  Interac¬ 
tive  ISD  Federation  will  provide  a 
mechanism  for  developing  and 
exercising  a  prototype  system  in  a 
simulated  environment.  This  will, 
in  effect,  create  a  laboratory  for 
trying  out  a  design  or  an  engineer¬ 
ing  change  proposal,  before  its 
approval  as  an  engineering 
requirement. 

•  Design  and  Analysis.  The  simula¬ 
tions  provide  a  mechanism  for  the 


collection  of  performance  data  as 
a  basis  for  design  of  system  modi¬ 
fications.  A  significant  feature  is 
the  ability  to  conduct  repeatable 
test  conditions,  and  the  capability 
to  parametrically  vary  the  condi¬ 
tions  in  a  controlled  manner. 

•  Testing  and  Evaluation.  The  ISD 
Federation  will  provide  a  virtual 
simulation  capability  that  will 
enhance  test  and  evaluation  efforts 
by  providing  better-designed  sys¬ 
tems  as  a  result  of  testing  earlier  in 
the  development  phase.  A  wider 
scope  of  testing  may  be  possible 
for  some  systems,  especially  those 
that  require  large  scenarios  of  cost¬ 
ly  test  services,  such  as  multiple 
aircraft  flyovers  or  test  targets  and 
associated  range  services. 

•  Doctrine  and  Tactics.  The  ISD 
Federation  will  provide  a  method 
to  evaluate  the  tactics  and  doctrine 
by  exercising  the  prototype  ISDS 
human-machine  interface  in  con¬ 
junction  with  the  simulated  sen¬ 
sors  and  weapons. 

Figure  5  Phase  III  Architecture 


Bringing  M&S  Into  Focus 

The  key  issue  for  program  managers 
to  understand  is  that  as  M&S  is 
brought  into  focus  DoD-wide,  the 
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realized.  Because  of  declining  bud¬ 
gets  and  technically  advanced  sys¬ 
tems,  we  can  no  longer  continue 
business  as  usual  and  expect  to  field 
the  same  quality  systems.  We  must 
rely  more  on  the  benefits  M&S  can 
provide,  but  first  we  need  to  lay  the 
foundation  that  makes  that  possible. 
Program  managers  need  to  have  a 
high  degree  of  confidence  in  their 
models  and  the  subject  matter 
experts  to  operate  them.  The  key  is 
to  get  started,  take  a  small  piece  of 
the  problems,  and  work  from  there. 
The  momentum  of  success  and 
opportunity  to  leverage  from  other’s 
work  will  carry  the  effort  forward. 
Every  effort  toward  this  goal  helps 
by  bringing  M&S  into  clear  focus  for 
the  acquisition  community. 
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Total  Value  Assessment  for  the 
acquisition,  delivery,  provision¬ 
ing,  and  sustainment  of 
warfighting  forces  requires  a 
greater  understanding  of  how 
these  processes  develop,  interrelate, 
and  evolve  in  real-world  situations. 
These  processes  are  complex  and  tend 
to  depend  upon  an  immense  amount 
of  data.  In  many  cases,  small  changes 
in  the  environment  produce  large  dis¬ 
continuous  changes  in  the  way  the 
processes  work.  For  example,  estimat¬ 
ing  the  amount  of  supplies  needed  for 
Desert  Storm,  actually  shipping  them, 
and  returning  the  unused  items  after¬ 
ward  stressed  our  ability  to  predict 
and  control  the  supply  process,  and 
resulted  in  quite  a  few  undesirable  rip¬ 
ple  effects. 

Vision 

Creating  models  for  acquisition-logis¬ 
tics  processes  and  simulating  their 
execution  within  a  synthetic  environ¬ 
ment  provides  the  best  tool  available 
for  assessing  the  total  value  of  prod¬ 
ucts  and  their  associated  processes. 
Within  a  synthetic  environment,  we 
can  instrument  and  monitor  an 
unfolding  process  and  its  constituent 
product(s)  to  gather  data  for  later 
analysis;  or  in  real  time,  interactively 
ask  “what-if”  questions  by  making 
adjustments  to  the  product(s)  and 
process(es)  to  better  understand  resul¬ 
tant  behavior. 
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A  major  DoD  Modeling  and  Simula¬ 
tion  (M&S)  objective  is  to  perform  vir¬ 
tual  warfare  engagements  using  simu¬ 
lated  and  actual  weapon  systems.  This 
vision  and  its  objectives  can  be  broad¬ 
ened  to  include  the  acquisition  and 
logistics  processes  of  simulated  and 
actual  systems.  For  example,  a  logis¬ 
tics  planning  exercise  using  weather 
and  climate  data  may  link  to  actual 
operational  supply  vessels  and  into 
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commercial  transport  systems  so  as  to 
assess  the  trades  of  augmenting  DoD 
systems  with  commercial  delivery  sys¬ 
tems.  Imagine  if  this  same  exercise 
included  connections  to  models  and 
simulations  of  the  transported  prod¬ 
ucts  so  the  issues  of  retrofit  and  manu¬ 
facturing  could  also  be  addressed,  giv¬ 
ing  logisticians  an  even  more  complete 
or  total  value  assessment  of  all 
options. 

Achieving  the  Vision 

Achieving  the  vision  of  integrated 
acquisition  and  logistics  synthetic 
environments  still  presents  a  number 
of  technology  challenges.  Before  vir¬ 


tual  engagements  can  be  of  most 
value  to  life-cycle  analysis,  they  will 
require  high-fidelity  system  models. 
During  the  course  of  analysis,  we 
need  an  ability  to  refine  components 
of  high-level  aggregate  models  into 
detailed  high-fidelity  models  to  better 
explore  specific  aspects  of  a  life-cycle 
problem.  M&S  is  already  used  for 
planning  and  warfare  analysis  at  dif¬ 
ferent  levels  of  abstraction  (cam¬ 
paign,  engagement,  and  system  inter¬ 
operation);  however,  current  M&S 
systems  have  little  ability  to  integrate 


multiple-fidelity  models  into  a  simu¬ 
lation  exercise. 

No  single  organization  will  be  able  to 
build  and  maintain  the  collection  of 
virtual  prototypes  needed  for  these 
exercises.  Since  prototypes  will  be 
built  and  used  by  many  different  organ¬ 
izations,  achieving  interoperability 
requires  the  use  of  at  least  de  facto 
standards  and  perhaps  an  organiza¬ 
tion  to  promulgate  those  standards. 
Currently,  some  standards  are  begin¬ 
ning  to  emerge  for  representing  virtual 
prototypes.  For  example,  modelers 
explicitly  designed  Virtual  Reality 
Modeling  Language  (VRML)  to  pro¬ 
duce  virtual  prototypes  that  can  be 
placed  in  synthetic  worlds  and  interact 
with  other  objects  in  these  worlds. 
This  emerging  technology  needs  to  be 
integrated  and  more  exploited  within 
the  acquisition-logistics  community. 

Constructing  and  performing  assess¬ 
ment  exercises  in  a  synthetic  environ¬ 
ment  requires  a  distributed  modeling 
and  simulation  framework  in  which  a 
user  can  discover  and  configure  virtual 
prototypes,  then  launch  exercises 
without  human  involvement  at  any  of 
the  distributed  sites  that  contain  pro¬ 
totypes.  Commercial  technology  and 
standards  that  address  tool-to-tool 
communication  (e.g.,  Common  Object 
Request  Broker  Architecture  [CORBA], 
Internet  protocols)  are  available  and 
can  be  exploited  for  the  assembly  of 
distributed  synthetic  environments. 

SBD’s  Influence/Accomplishments 

The  Defense  Advanced  Research  Pro¬ 
jects  Agency  Simulation  Based  Design 
(SBD)  program  is  developing  a  proto¬ 
type  distributive,  collaborative  soft¬ 
ware  system  that  addresses  some  of 
the  functions  required  for  fielding  the 
types  of  integrated  acquisition-logistics 
synthetic  environments  that  support 
the  development,  analysis,  and  inter¬ 
operation  of  virtual  prototypes. 

Previously,  DARPAs  SBD  program  vali¬ 
dated  the  feasibility  of  establishing 
distributed  synthetic  collaboration 
environments  between  multiple  het¬ 
erogeneous  organizations.  To  meet 


new  threats,  these  collaborative  syn¬ 
thetic  environments  used  engineering 
analysis  to  better  evaluate  operational 
warfighting  performance  and  used 
operational  analysis  to  reengineer 
weapon  systems. 

Engineers  also  used  SBD  to  develop 
conceptual  design  models  and 
detailed  engineering  models  for  ships. 
Of  sufficient  structural  detail  that 
modelers  can  use  them  for  parametric 
design  optimization,  the  conceptual 
models  can  be  placed  in  high-fidelity 
operating  environments.  The  detailed 
models  have  been  used  to  generate 
shop  floor  manufacturing  instructions 
and  to  provide  immersive  maintenance 
training. 

During  the  past  year,  the  SBD  program 
performed  a  validation  experiment, 
called  the  Advanced  Surface  Combat¬ 
ant  (ASC),  that  culminated  in  a  Febru¬ 
ary  1997  demonstration  of  SBD  matu¬ 
rity.  This  experiment  specifically 
focused  on  the  survivability  analysis 
and  redesign  of  a  surface  combatant  to 
meet  a  new  threat.  It  also  provided  the 
opportunity  to  include  detailed 
physics-based  models  in  the  warfight¬ 
ing  analysis  phase  and  use  of  multidis¬ 
ciplinary  optimization  techniques  to 
provide  parametric  design  information 
to  the  redesign  process. 

The  ASC  Experiment  resulted  in  an 
SBD  system  configuration  that  — 

•  integrated  multiple  companies  and 
government  agencies  into  an  Inte¬ 
grated  Product  Team  (IPT); 
•organized  the  IPT  as  a  hierarchical 
collection  of  federations; 
•operated  over  a  combination  of 
Local  Area  Network,  Internet,  and 
DARPA  gigabit  testbed  (ATDNet) 
network  resources; 

•integrated  approximately  30  soft¬ 
ware  components  into  the  system; 
•integrated  two  legacy  databases 
and  ingested  the  indicative  design 
of  the  Navy  SC-21; 

•provided  interface  code  that 
wrapped  legacy  simulations  mak¬ 
ing  them  compliant  with  the 
DMSO  High  Level  Architecture; 
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Figure  1  Product  Life-Cycle  Activities,  Work  Products,  and  Impacts 


•  demonstrated  the  use  of  SBD  in 
multiple  life-cycle  activities  (from 
requirements  to  training); 

•  demonstrated  the  use  of  multidis¬ 
ciplinary  analysis  and  optimiza¬ 
tion;  and 

•incorporated  cost  as  an  indepen¬ 
dent  variable  in  the  design  trades. 

The  ASC  experiment  represented  a 
significant  achievement  in  maturing 
SBD  technology  to  the  stage  that  it  can 
now  be  deployed  for  experimental  use 
by  contractors  and  the  government  for 
conceptual  system  design,  develop¬ 
ment,  and  evaluation. 

The  SBD  Product 

The  SBD  system  is  a  collaborative, 
multidisciplinary  environment  for 
developing  and  using  virtual/real  pro¬ 
totypes.  Engineers  configure  an  SBD 
system  for  a  specific  application  by 
linking  copies  of  a  common  set  of  soft¬ 
ware,  called  the  Core  Processing  Sys¬ 
tem,  together  with  application-specific 
software  tools.  SBD  allows  engineers 
to  develop,  analyze,  and  operate  virtu¬ 
al  prototypes  as  they  would  actual 
prototypes,  but  without  the  cost  and 


complexity  associated  with  real  hard¬ 
ware  and  materials.  A  virtual  prototype 
is  a  computer  software  module  that 
models  the  structure  and  behavior  of 
the  actual  product  under  develop¬ 
ment.  The  process  of  producing  an 
actual  product  proceeds  as  a  series  of 
virtual  prototypes  that  defines  the 
product  and/ or  generates  manufactur¬ 
ing  instructions  for  the  product.  For 
the  virtual  prototyping  process  to  yield 
actual  quality  products,  the  same  dis¬ 
ciplines  must  be  applied  in  the  virtual 
prototyping  process  as  are  applied  in 
conventional  product  development 
processes. 

Numerous  integrated  development 
environments  exist,  tailored  to  a 
selected  computer-aided  design  (CAD) 
tool  (e.g.,  the  Boeing  777  CATIA™- 
based  environment),  and  many  orga¬ 
nizations  have  now  integrated  model¬ 
ing,  simulation,  visualization,  and 
analysis  tools  for  product  develop¬ 
ment.  However,  engineers  craft  these 
concurrent  engineering  systems  for 
specific  applications,  which  signifi¬ 
cantly  limits  their  reusability,  even 
between  different  projects  in  the  same 


organization.  Developing  the  second 
system  becomes  as  expensive  as  devel¬ 
oping  the  first.  Further,  there  are  no 
standards  to  allow  these  different  sys¬ 
tems  to  interoperate. 

The  SBD  process  employs  a  much 
more  open  approach  that  produces  a 
variety  of  design,  engineering,  and 
evaluation  results.  Figure  1  illustrates 
the  product  development  activities, 
SBD  work  products,  and  their  impact 
on  the  product  life  cycle.  This  process 
delivers  better  quality  products  at  a 
reduced  cost,  risk,  and  schedule  when 
compared  to  the  current,  more  con¬ 
ventional  concurrent  engineering 
approaches.  The  virtual  prototyping 
activities  can  be  conducted  in  a  dis¬ 
tributed  collaborative  software  envi¬ 
ronment,  which  allows  more  concur¬ 
rence  in  the  development  tasks,  thus 
reducing  schedule  slippage. 

Using  virtual  prototypes  for  engineer¬ 
ing  analysis  and  operational  validation 
also  allows  for  investigation  of  larger 
solution  spaces.  Changes  can  be  made 
much  later  in  the  virtual  product 
development  life  cycle  without  incur- 
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Figure  2.  SBD  Integrates  Multidisciplinary  Life-Cycle  Activities 


ring  the  cost  magnitude  that  changes 
further  downstream  make  in  conven¬ 
tional  development  processes. 

Figure  2  illustrates  the  life-cycle  activities 
for  a  notional  ASC  Navy  program,  inte¬ 
grated  and  supported  by  a  collaborative 
SBD  system  configured  as  an  IPT.  The 
IPT  involved  multiple  government  and 
contractor  organizations  with  partici¬ 
pants  for  program  management,  design, 
engineering  analysis,  operational  test  and 
evaluation,  and  deployment  and  training 

The  ASC  SBD  system  linked  multiple 
copies  of  the  common  software  com¬ 
ponents  together  to  support  the  IPT. 
Engineers  configured  each  partici- 
pants  software  to  reflect  one  of  the  fol¬ 
lowing  four  roles  in  the  life-cycle  devel¬ 
opment  process: 

•  Program  Management  Office 
•Hull  Mechanical  and  Electrical 

Design 

•  Combat  System  Design 

•  Survivability  Analysis 

While  some  of  the  ASC  software  was 
ship-specific,  much  was  domain-inde¬ 


pendent  and  could  be  used  for  other 
application  areas.  The  experience  of 
integrating  such  an  SBD  system  trans¬ 
lates  readily  to  other  domains. 

Using  SBD 

With  SBD,  a  user  can  define,  modify, 
visualize,  and  manipulate  virtual  prod¬ 
ucts.  The  SBD  system  coordinates  the 
management  between  multiple  user 
activities  by  using  virtual  prototypes 
that  are  composed  as  assemblies  of 
subsystems  and  parts.  Engineers 
define  the  actual  construction  of  parts 
in  terms  of  material,  structure,  and 
behavior  attributes.  By  combining 
legacy  models  in  various  ways  and  by 
producing  data  that  can  be  used  by  a 
variety  of  legacy  analysis  tools,  they 
construct  virtual  products.  The  values 
then,  of  these  attributes  may  be  com¬ 
puted  by  external  tools  or  incorporat¬ 
ed  from  legacy  databases. 

Users  access  SBD  through  a  standard 
web  browser.  Figure  3  shows  a  satellite 
prototype  as  viewed  from  an  early  SBD 
User  interface  prototype.  Since  a  key 
feature  of  the  user  interface  is  its  use 
of  standard  web  browsers,  it  can  easily 


use  standard  plug-in  tools  such  as 
VRML  viewers  to  display  a  wide  spec¬ 
trum  of  standard  data  types. 

This  particular  user  interface  proto¬ 
type  lacks  the  elements  for  controlling 
analysis  and  design  tools,  but  it  does 
show  how  engineers  can  easily  access 
information  about  the  design  ele¬ 
ments.  In  this  example,  the  window 
on  the  right  shows  a  component  hier¬ 
archy  of  the  satellite  and  allows  the 
user  to  access  components  like  bus 
structure,  power,  propulsion,  attitude 
determination  and  control  system, 
thermal,  and  payload  modules.  Each 
of  these  components  has  its  own 
decomposition,  and  the  subcompo¬ 
nents  are  interconnected  in  various 
ways.  Connections  are  maintained  as 
part  of  the  product  definition. 

The  window  on  the  left  displays  the 
satellite  as  viewed  within  a  3D  visual¬ 
ization  and  interaction  environment. 
This  satellite  prototype  responds  to  a 
set  of  commands  that  can  be  used  to 
deploy  its  solar  panels  and  actuate 
mechanical  devices  on  the  satellite. 
Operating  the  satellite  within  this  kind 
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Figure  3  Satellite  Prototype  Viewed  from  the  SBD  Guided  User  Interlace  (GUI) 


of  environment  can  be  used  to  evalu¬ 
ate  a  number  of  design  properties 
such  as  sensor  field  of  view,  inter-satel¬ 
lite  communications,  ground  station 
communications,  mechanical  interfer¬ 
ence  properties  of  deployment  and 
actuation  devices,  and  advanced  tech¬ 
nology  insertion.  In  this  example, 
engineers  used  VRML  to  produce  the 
visual  appearance  of  the  satellite,  with 
the  VRML  being  computed  from  the 
virtual  product  representation. 

The  specifics  upon  which  modelers 
define  virtual  prototypes  differ, 
depending  on  the  level  of  fidelity 
needed  and  the  data  requirements  of 
the  tools  used  to  evaluate  the  proto¬ 
type.  Easy  to  modify  and  clone,  engi¬ 
neers  can  later  reuse  virtual  prototypes 
as  parts  of  other  prototypes.  New 
attributes  (e.g,  center  of  gravity)  can 
also  be  added  to  a  prototype  when 
needed  for  a  particular  application. 

Further,  engineers  can  quickly  generate 
an  initial  conceptual  design  to  validate 
feasibility  and  provide  a  basis  for  cost 
estimation  by  reusing  data  from  previ¬ 
ous  systems  and  by  importing  data 
from  external  tools.  The  virtual  prod¬ 
ucts  can  then  be  analyzed  with  existing 
or  legacy  analysis  tools,  and  can  be 


operated  in  virtual  environments  com¬ 
bining  real  and  simulated  products. 

With  SBD,  engineers  can  also  capture 
design  processes,  such  as  the  steps  in 
designing  a  power  subsystem  for  the 
satellite,  as  mega-programs  —  or  pro¬ 
grams  of  programs  -  that  are  manipu¬ 
lated  and  operated  exactly  like  the  vir¬ 
tual  prototypes. 

Development  proceeds  within  SBD  by 
establishing  product  constraints  and 
requirements  and  by  constructing 
increasingly  detailed  virtual  prototypes 
of  the  product.  The  virtual  prototypes 
(software  models)  can  be  viewed, 
interacted  with,  analyzed,  and  operat¬ 
ed  like  real  prototypes.  As  engineers 
make  design  changes,  analysis  and 
evaluation  of  the  prototypes  takes 
place  within  synthetic  physics-based 
environments.  SBD  not  only  manages 
these  design  artifacts,  with  built-in 
configuration  management  tools,  but 
also  allows  engineers  to  incorporate 
components  of  different  levels  of 
fidelity  within  a  virtual  prototype. 

Collaborating  With  SBD 

Complex  product  development  typi¬ 
cally  involves  multiple  heterogeneous 
organizations.  The  inter-connectivity 


needed  for  product  development 
requires  support  for  defining,  manag¬ 
ing,  and  enforcing  development 
processes  and  the  resulting  workflow. 
In  a  large-scale  product  development 
enterprise,  each  team  has  its  own  data, 
product,  and  process  models. 

Since  engineers  configure  a  collabora¬ 
tive  SBD  system  as  a  collection  of 
copies  of  the  common  software,  it 
works  to  provide  seamless  access  to  all 
public  resources  in  the  entire  SBD  sys¬ 
tem.  Each  user  interface  provides 
access  to  the  rest  of  the  SBD  system,  as 
mediated  by  the  Core  Processing  Sys¬ 
tem  components. 

SBD  allows  users  to  maintain  their  prod¬ 
uct  data  in  one  or  more  databases  —lega¬ 
cy  or  new,  flat  file,  relational,  or  object  ori¬ 
ented  —  which  can  be  either  centralized 
or  distributed.  Each  Core  Processing  Sys¬ 
tem  maintains  an  object  model  that  is 
accessed  from  its  user  interface,  for  visu¬ 
alization  and  interaction. 

A  Project's  View  of  SBD 

SBD  uses  object  models  to  represent 
all  product  and  process  information. 
Within  an  enterprises  organizational 
hierarchy,  object  models,  collectively 
called  the  Smart  Product  Model 
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Figure  4.  A  Logical  View  of  the  SPfl 


(SPM),  define  and  manage  the  devel¬ 
opment  process,  the  external  soft¬ 
ware  and  databases  that  are  part  of  a 
specific  SBD  configuration,  as  well  as 
the  product  models  that  engineers 
develop  as  part  of  the  life-cycle 
process. 

The  SPM  may  be  viewed  as  a  collection 
of  concentric  circles  with  data  at  the 
center;  “smart”  methods  that  directly 
operate  on  the  data  in  the  innermost 
ring;  software  components  that  pro¬ 
vide  value-added  services  such  as  3D 
viewing  and  interaction  with  product 
model  data  as  the  next  ring;  and  final¬ 
ly,  external  programs  that  interface  to 
the  core  data  as  the  outer  ring. 

Figure  4  illustrates  this  view  of  the 
architecture  for  a  notional  ship  design 
project. 

This  architecture  is  a  natural  exten¬ 
sion  of  the  single  CAD  model 
approach  used  on  large  programs 
such  as  Boeing  777.  Integrating 
behavior,  management,  and  analysis 
data  into  a  single  virtual  enterprise¬ 
wide  distributed  data  model  ensures 
that  all  members  of  the  team  always 
have  access  to  all  information  relevant 
to  their  design,  and  that  the  impact  of 


design  or  management  changes  — 
such  as  schedules  or  budgets  —  can 
be  immediately  assessed  by  all  team 
members. 

The  object  models  are  “smart”  because 
they  have  methods  that  are  used  to 
perform  analysis  and  other  develop¬ 
ment  activities.  Methods  are  the 
means  to  manipulate  or  analyze  data 
such  as  meshing  of  CAD  data  for 
structural  analysis,  aerodynamics  for 
aircraft  maneuverability,  or  a  seakeep¬ 
ing  model  for  ship  motions.  As  such, 
they  can  be  aggregated  to  form  views 
into  the  object  model  that  are  specific 
to  a  given  discipline  or  user  group.  As 
an  example,  methods  may  be  used  to 
calculate  the  weight  of  an  object  as  the 
sum  of  the  weights  of  its  components; 
or  methods  can  be  used  to  expose  a 
data  view  relevant  to  the  structural 
design  engineer. 

Conclusion 

SBD  is  the  first  step  toward  fielding 
integrated  acquisition-logistics  syn¬ 
thetic  environments.  By  harnessing 
advancements  in  M&S,  High  Perfor¬ 
mance  Computing  and  Communica¬ 
tions,  and  Multimedia  technologies, 
SBD  provides  a  virtual  collaborative 
environment  for  geographically  dis¬ 


tributed  IPTs  to  design  complex  sys¬ 
tems  and  provide  support  throughout 
the  product’s  life  cycle. 

Today,  M&S  is  becoming  increasingly 
important  in  acquiring  systems  for  the 
government,  but  the  potential  cost 
reductions  offered  by  correcdy  using 
M&S  in  the  design  process  still  dwarf 
deployed  reality.  Why  hasn’t  the  DoD 
acquisition  community  yet  realized 
these  substantial  cost  reductions?  The 
answer  is  contained  in  the  following 
three  problem  areas:  tools  don’t  interop¬ 
erate,  people  are  in  the  loop  even  when 
no  decision-making  requirements  exist, 
and  no  standards  for  digital  product  and 
process  models  exist.  SBD  offers  a  solu¬ 
tion  to  these  problems  by  leveraging 
emerging  standards  and  commercial 
forces  for  interoperability,  by  fielding  a 
collaborative  software  environment  infra¬ 
structure,  and  by  creating  de  facto  stan¬ 
dards  for  product  and  process  models. 

The  SBD  program  is  unique  in  develop¬ 
ing  a  virtual  prototyping  architecture  for 
configuring  reusable  and  interconnec¬ 
tive  SBD  systems  with  standards-based 
interfaces.  The  ability  to  reuse  and  inter¬ 
operate  virtual  products  across  multiple 
organizations  and  vendors  is  where  SBD 
will  pay  off  on  a  national  scale. 
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MODELING  AND  SIMULATION 


Why  is  Modeling  and  Simulation 
So  Hard  to  Do? 

FI&S  Commonalities,  Interoperable  Systems  Will 
Provide  Warfighters,  Decision  Nakers  Increased 
Readiness  Across  Full  Spectrum  of  Conflict 


“No  one  knows  exactly  what  warfare  in 
the  21st  Century  will  be  like .  However, 
one  thing  is  certain  -future  battlefields 
will  be  far  different  and  more  complex 
than  20th  Century  battlefields.  We  must 
be  ready... Finding  ways  to  exploit  our 
competitive  advantages  -  quality  people 
and  advancing  technology  -  becomes  our 
future  readiness  challenge 

-Gen.  Dennis  J.  Reimer 
Army  Chief  of  Staff 

The  issue  of  Simulation  Based 
Acquisition  poses  an  interesting 
dichotomy  for  the  Defense 
Department  and  its  support 
industries.  On  the  one  hand,  it 
holds  the  potential  to  be  the  greatest 
tool  to  improve  the  acquisition 
process;  but  on  the  other  hand,  the 
number  of  systems  and  programs 
using  simulation  in  new,  innovative 
ways  are  few  and  far  between.  Given 
the  enormous  pressure  to  reduce 
costs,  save  time,  and  make  innovative 
uses  of  technology  in  all  facets  of  our 
lives,  why  does  this  obvious  area  of 
need  seem  to  be  lagging  behind?  It 
appears  that  the  difficulties  are  not 
technological  as  much  as  they  are  cul¬ 
tural,  organizational,  and  yes,  even  a 
function  of  policy 

The  Stated  Need 

The  use  of  modeling  and  simulation 
(M&S)  in  the  military  and  its  support¬ 
ing  industries  is  increasing.  As  these 
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needs  increase,  the  demand  for  non¬ 
technical  personnel  to  provide  man¬ 
agement  and  leadership  also  increases. 
The  senior  leadership  of  each  Service 
express  these  needs  in  their  individual 
M&S  plans. 

The  Army  Model  and  Simulation  Mas¬ 
ter  Plan1  promotes  the  adoption  of 
M&S  standards,  common  tools,  and 
processes  for  use  in  all  applications 
throughout  the  Army.  In  an  effort  to 
invest  its  resources  in  an  effective  and 
efficient  fashion,  the  Department  of 
the  Army  intends  to  use  M&S  tech¬ 
nologies  to  significantly  advance  the 
capabilities  of  a  smaller,  power-projec¬ 
tion  Army  capable  of  land  force  domi¬ 


nance.2  The  Master  Plan  requires  that 
the  Army  seek  opportunities  for  com¬ 
monality  within  M&S  technologies 
and  capitalize  upon  them,  wherever 
feasible. 

The  U.S.  Air  Force  (USAF)  Modeling 
and  Simulation  Master  Plan  states  the 
Air  Force  goal  for  M&S  is  to  develop  a 
capability,  using  interoperable  M&S 
systems,  to  provide  warfighters  and 
decision  makers  the  tools  to  ensure 
readiness  across  the  full  spectrum  of 
conflict.3  Fully  capable  of  supporting 
analysis  and  training,  which  is  inte¬ 
grated  throughout  all  echelons  of 
the  Air  Force,  the  Air  Force  M&S 
architecture  links  together  many 
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types  of  simulations  (e.g.,  aggregate 
and  detailed  computer  models,  pilots 
in  live  aircraft  and  simulators,  and 
hardware  components). 

The  Air  Force  has  always  used  models 
and  simulations  of  reality,  considering 
live  field  exercises  as  simulated  war¬ 
fare4 

The  U.S.  Marine  Corps  (USMC) 
desires  to  acquire  and  apply  M&S 
technologies  effectively  and  efficiently 
to  support  USMC  roles  and  missions.5 
Recognizing  that  the  use  of  M&S 
enhances  training,  education,  analysis, 
logistics,  planning,  and  the  conduct  of 
operations,  the  USMC  also  promotes 
the  use  of  M&S  as  the  very  basis  for 
improving  future  acquisition  deci¬ 
sions,  systems  testing  and  evaluation, 
realignment  of  force  structure,  and 


warfighting  capability  by  exploiting 
world  class  M&S  technology  in  order 
to  take  full  advantage  of  the  explosion 
in  information  and  communications 
technologies,  thereby  improving  Total 
Force  performance.  By  ensuring  that  it 
simulates  before  it  builds,  buys,  or 
fights,  the  Corps  will  enhance  readi¬ 
ness  and  training  while  simultaneous¬ 
ly  reducing  costs.7 

The  Department  of  the  Navy  (DON) 
has  stated  it  will  use  the  appropriate 
level  of  M&S  in  order  to  support  all 
phases  and  milestone  decisions  of  the 
system  acquisition  cycle.8  The  end- 
state  objectives  of  the  Navy’s  M&S 
plan  includes  a  full-scale  integration  of 
live,  virtual,  and  constructive  simula¬ 
tion  into  training  endeavors,  and  the 
enabling  of  mission  planning  and 
rehearsal  through  the  use  of  M&S.9 


community.  For  years,  senior  acquisi¬ 
tion  leaders  throughout  the  Depart¬ 
ment  of  Defense  (DoD)  discussed  a 
future  goal  of  streamlining  the  acquisi¬ 
tion  process.  For  people  outside  the 
military  [and  oftentimes,  inside],  the 
acquisition  life  cycle  is  almost  unbe¬ 
lievable.  For  example,  the  Air  Force 
began  work  on  the  F-14  as  early  as 
1961,  the  Ml  Tank  in  1969,  and  the 
Stealth  Fighter  in  1978;  in  fact,  an  aver¬ 
age  acquisition  life  cycle  of  15  years  for 
even  small  systems  is  not  unusual.  The 
need  to  streamline  is  great,  and  the 
process  has  many  points  that  would 
seem  to  warrant  some  technological 
improvements.  Let’s  look  at  a  few. 

Concept  Formulation/Defining  Re¬ 
quirements.  We’re  all  familiar  with  the 
cartoon  that  shows  the  series  of  events 
illustrating  how  the  camel  evolved  via 


requirements  defini- 


The  Marine  Corps 
Modeling  and  Simula¬ 
tion  Master  Plan 
states  that  the  Marine 
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Photo  courtesy  Lockheed  Martin  Corporation 

"Fop  people  outside  the  military  [and  oftentimes, 
inside],  the  acquisition  life  cycle  is  almost  unbe-  ' 
uevable  For  example,  the  Air  Force  began  work 
ON  THE  F-14  AS  EARLY  AS  1961,  THE  Ml  TANK  IN 
:  1969,  and  the  Stealth  Fighter  in  1978;  in  fact,  an 

AVERAGE  ACQUISITION  LIFE  CYCLE  OF  15  YEARS  FOR  EVEN 
SMALL  SYSTEMS  IS  NOT  UNUSUAL" 


The  Solution 

The  intent  of  all  the  Services 
and,  in  many  cases,  the  Con¬ 
gress,  seems  quite  clear,  and 
many  of  us  believe  that  the 
domain  of  M&S  that  could 
gain  the  most  from  this  new 
technology  is  the  acquisition 


the  acquisition  process,  when  a  horse 
was  the  original  concept.  Although  a 
trite  example,  it  does  typify  what  we 
all  experienced,  as  the  user  first 
describes  the  need  and  then  passes  it 
to  the  developer,  who  must  then  con¬ 
vert  the  idea  into  the  best  technical 
solution.  The  challenge  is  for  the  user 
to  initially  communicate  the  needed 
system  in  operational  terms,  while  the 
developer  must  design  and  develop 
something  that  meets  the  needs  in 
terms  of  a  real,  efficient,  and  maintain¬ 
able  item  of  equipment. 

Further,  this  is  often  complicated  by 
language  problems,  personnel  turn¬ 
over,  technology  changes,  priority 
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changes,  and  leadership  directions.  Of 
course,  the  real  problem  is  that  users 
really  have  a  difficult  task  describing 
what  the  new  requirement  is;  by 
nature,  they  want  everything,  they 
want  it  today,  and  they  want  it  cheap! 
Who  can  blame  them  when  they  are 
representing  the  needs  of  the  warfight¬ 
ers,  who  are  always  faced  with  new 
missions  and  bigger  challenges.  The 
problem  is  that  this  often  ambitious, 
yet  less-than-detailed  Operational 
Requirements  Document  (ORD)  is 
very  difficult  for  developers  to  imple¬ 
ment.  In  addition,  as  modelers  devel¬ 
op  many  of  the  capabilities,  technical 
solutions  often  end  up  as  useful  but 
not  consistent  with  the  original 
requirement.  This  is  not  always  recog¬ 
nized,  as  the  documentation  of  the 
original  need  is  not  usually  available  to 
the  developers.  A  very  long  trail, 
indeed.... 

Documentation.  When  the  acquisi¬ 
tion  of  the  training  subsystem  alone 
includes  a  trailer  truckload  of  docu¬ 
ments,  it  becomes  easy  to  understand 
why  the  documentation  of  the  acquisi¬ 
tion  life  cycle  is  so  difficult  to  manage 
and  often  lags  behind  when  develop¬ 
ment  work  becomes  overwhelming. 
Certainly,  modern  information  tech¬ 
nology  can  alleviate  this  problem,  sim¬ 
ply  by  automating  the  existing  com¬ 
plex  “paper”  process.  Making  use  of 
the  current  techniques  of  distributed 
data  systems,  electronic  conferencing, 
and  Web-based  document  collabora¬ 
tion  would  provide  not  only  a  ready 
access  to  the  ORD,  but  also  provide  an 
online  ability  to  document  decisions 
and  actions  throughout  the  process. 
The  idea  that  one  phase  of  the  process 
could  pass  its  experience  on  to  the 
next,  including  issues  that  need  reso¬ 
lution  and  key  decisions  that  help 
accomplish  the  requirement,  would 
reduce  the  time  and  transfer  loss  that 
happens  at  each  milestone.  The  use  of 
consistent  state-of-the-art  information 
technology  alone  would  reduce  the 
process  by  15-20  percent. 

Simulation  in  Defining  Concepts 
and  Development  of  the  ORD.  Re¬ 
member  the  hardest  thing  about  doing 


a  term  paper  in  high  school?  Most  of 
us  would  probably  reply  that  it  was 
determining  the  topic  and  theme  of 
the  paper.  This  process  is  similar  to 
trying  to  describe  the  functional  capa¬ 
bilities  of  a  new  weapons  system, 
which  has  become  especially  difficult 
with  the  transition  from  a  require- 
ments-based  system  to  a  capability- 
based  approach.  One  promising  alter¬ 
native  approach  that  uses  simulation  is 
the  development  of  a  notional  system 
using  a  dynamic  computer  model,  at 
the  component  level  of  the  systems. 
Modelers  would  begin  by  first  loading 
the  system  that  currendy  exists  into  a 
computer  simulation  that  can  dynami¬ 
cally  and  graphically  display  the 
appearance  and  performance  capabili¬ 
ty  of  the  components  that  make  up 
the  system.  Depending  on  the  com¬ 
plexity  of  the  system,  it  could  end  up 
being  a  multi-level  model,  consisting 
of  “system  of  systems.”  As  most  sys¬ 
tems  are  actually  only  about  25  per¬ 
cent  new  technology,  the  combat 
developer  systematically  works 
through  each  major  sub-system, 
replacing  components  with  either 
existing  components  from  other  mili¬ 
tary  or  civilian  systems,  or  defining  a 
new  system  based  on  functional  capa¬ 
bility.  Plugging  the  new  items  in,  of 
course,  must  include  a  reconfiguration 
of  support  systems  and  recalibration 
of  performance  parameters. 

Once  the  developer  completes  the 
functional  virtual  prototype,  initial 
operational  testing  comes  next  to 
determine  the  prototype’s  perfor¬ 
mance  capabilities.  By  injecting  the 
Virtual  Prototype  into  a  battle  sce¬ 
nario,  previously  baselined  with  the 
existing  system,  modelers  can  then  see 
if  they  are  achieving  the  desired  out¬ 
comes.  Data  can  be  collected  for  those 
components  that  are  real,  and  can  be 
approximated  for  the  completely  new 
pieces.  Once  the  concept  is  tried  out, 
the  performance  parameters  and  the 
documented  functionality  can  be 
translated  into  an  ORD,  and  the  virtu¬ 
al  prototype  can  be  passed  on  to  the 
developer  to  ensure  proper  under¬ 
standing  of  the  requirements  and 
maintenance  of  all  the  information 


generated  up  to  this  point.  Of  course, 
key  to  this  process  is  ensuring  that 
modelers  use  the  new  concepts  in 
such  a  way  that  performance  can  be 
accurately  measured  and  evaluated  in 
terms  of  system  and  sub-system  per¬ 
formance,  as  well  as  operational  and 
tactical  ability.  This  takes  us  to  our 
next  streamlining  opportunity. 

Test  and  Evaluation.  Easily  the  most 
underutilized  element  of  the  overall 
acquisition  community,  test  and  evalu¬ 
ation  could  provide  25-  to  40-percent 
savings  if  properly  employed  through¬ 
out  the  life  cycle  process.  In  the  first 
place,  most  programs  wait  until  the 
end  to  begin  involving  the  test  and 
evaluation  (T&E)  community  when, 
in  fact,  the  T&E  experts  should  be  on 
board  from  the  very  beginning.  First, 
at  the  onset  of  the  concept  formula¬ 
tion  process,  the  T&E  experts  -  who 
understand  data  collection,  perfor¬ 
mance  assessment,  and  measures  of 
effectiveness  —  can  assist  in  the  formu¬ 
lation  process  by  pointing  out  those 
processes  already  tried,  and  those  that 
cannot  be  accurately  measured,  as 
described.  As  the  concept  is  converted 
into  a  prototype  (hopefully,  a  virtual 
prototype  as  described  previously), 
the  T&E  experts  can  help  set  up  ways 
to  measure  the  effectiveness  of  the  pro¬ 
totype,  as  well  as  set  up  and  measure 
the  test  program  against  the  current 
baseline  system.  In  some  instances, 
they  can  provide  facilities  or,  at  the 
very  least,  insight,  into  how  to  conduct 
virtual  tests,  and  can  even  do  sophisti¬ 
cated  hardware-in-the-loop,  engineer¬ 
ing-level  developmental  testing.  At  the 
same  time,  they  can  develop  the  test 
process  so  that  data  collected  can  be 
used  for  two  other  key  elements  relat¬ 
ed  to  Operational  Testing  -  Verifica¬ 
tion,  Validation,  and  Accreditation; 
and  cost  effectiveness.  They  should 
also  be  able  to  assist  in  leveraging  data 
from  previous  developmental  tests  on 
notional  components  from  other  test 
activities,  further  reducing  the  need  for 
testing.  When  this  is  coupled  with 
information  technology  automation 
techniques,  and  information  on  test 
experience  begins  flowing  between 
agencies  using  and  reusing  compo- 
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nent-level  data  and  evaluation  tools, 
the  process  becomes  more  efficient, 
and  the  life  cycle  becomes  shorter. 

A  lack  of  valid  data  to  use  in  the  mod¬ 
els,  and  the  lack  of  facility  most  of  us 
continue  to  have  in  truly  working  with 
data-intense  decisions,  constitute  two 
of  the  most  basic  reasons  simulation  is 
not  easier  to  implement.  Pound  for 
pound,  the  T&E  community  has  lived 
in  this  world  much  longer  than  the 
rest  of  us,  and  we  could  benefit  greatly 
from  their  experience. 

Other  Issues.  Certainly,  I  could  go  on 
and  talk  about  other  areas  that  could 
benefit  from  M&S  technology  inter¬ 
ventions.  These  could  include  the  use 
of  simulations  for  setting  up  virtual 
production  lines;  determining  parts 
needs  and  stock  levels;  using  simula¬ 
tion  to  simultaneously  develop  the 
necessary  training  systems;  using  the 
same  notional  approach  described 
previously  with  its  resultant  data  trail 
to  forecast  RAM  and  logistics  support 
and  using  a  mix  of  the  predecessor 
data  and  information  available  for  the 
components  connected  together.  This 
discussion  could  go  on  for  quite  some 
time.  However,  the  examples  I  just 
cited  should  be  enough  to  make  the 
point  that  the  use  of  simulation  in 
acquisition  is  not  a  mysterious 
process,  but  rather  the  managed  sys¬ 
tematic  integration  of  a  new  set  of 
technology  tools,  in  an  innovative  fash¬ 
ion.  But,  a  few  stumbling  blocks, 
which  are  not  technical  but  rather  cul¬ 
tural  and  organizational,  may  impede 
the  way. 

The  Problem 

Presendy,  no  focused,  organizational 
method  exists  that  ensures  individuals 
are  versed  in  the  issues  and  methods 
surrounding  M&S  applications  except 
by  on-the-job  training.  Even  within 
academia,  only  a  few  graduate  degree 
programs  in  Simulation  Systems  are 
offered.10’11  Despite  this  apparent  lack 
of  formal  training  and  education,  the 
need  for  DoD’s  expanded  use  of  M&S 
continues  to  be  viewed  as  a  major 
solution,  for  the  acquisition  world  and 
its  activities  continue  to  grow  at  a  sig- 


M&S  is  used 
everywhere  in  the  Air 
Force  because  better 
decisions  and  better 
training  make  better 
warfighters. 
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nificant  rate.  Without  a  formal  strategy 
for  developing  M&S  professionals, 
neither  consistent  application  nor 
functional  standardization  within  the 
M&S  community  can  be  achieved, 
and  acquisition  will  continue  to  go  on 
as  usual. 

In  addition,  until  the  Federal  Acquisi¬ 
tion  Regulation  changes,  many  of  the 
steps  and  streamlining  options  are,  in 
fact,  not  allowed.  Unless  program 
managers  receive  sufficient  latitude  to 
employ  these  alternative  techniques 
without  the  expectation  that  they 
must  solely  endure  the  pain  and 
shoulder  the  risks,  on  those  occasions 
when  the  fledgling  technology  fails, 
they  will  not  take  the  risk.  Only  when 
the  Departments  sponsor  key 
programs  to  do  some  classic  side- 
by-side  comparisons  of  applications 
using  simulation  versus  traditional 
approaches,  can  the  new  technologies 
prove  they  will  work,  saving  time  and 
money.  Then  it  will  be  possible  to  see 
Simulation  Based  Acquisition  achieve 
its  essential  role. 

Let  me  briefly  take  you  back  in  history 
a  few  years.  At  the  risk  of  sounding 
trite,  our  civilization  is  just  beginning 
to  shift  from  the  Industrial  Age  to  the 
Information  Age;  we  are  going  through 
all  the  dynamic  and  sometimes  painful 


processes  of  change.  If  we  look  at  how 
long  it  took  our  culture  to  go  through 
the  Industrial  Revolution,  we  can 
imagine  what’s  in  store  for  us.  Shifting 
from  a  focus  on  products  and  assem¬ 
bly-line  thinking  to  information  ser¬ 
vices  and  distributed  collaboration, 
will  clearly  be  a  large  leap.  Planners, 
modelers,  program  managers,  product 
managers  —  for  many  in  our  acquisi¬ 
tion  workforce,  this  shift  in  focus  may 
not  seem  efficient  or  pleasant. 

When  we  add  these  issues  to  the  chal¬ 
lenges  resulting  from  the  end  of  the 
Cold  War  and  the  huge  push  to 
expand  to  “operations-other-than-war” 
missions,  our  culture  is  going  through 
an  era  that  makes  the  ’60s  look  posi¬ 
tively  calm.  Only  by  systematic  plan¬ 
ning  and  careful  application  of  new 
technologies,  with  an  eye  always 
toward  the  best  outcome,  can  this 
process  be  streamlined  and  acquisi¬ 
tion  become  one  of  the  domains  that 
makes  full  use  of  available  technology. 
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Air  Force  Modeling  and 
Simulation  Trends 

Modeling  and  Simulation  Makes  Possible  the 
Unaffordable 
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Figure  1  fl&Sin  the  Air  Force  Enterprise 


Nodeling  and  Simulation 
(M&S)  is  already  an  integral 
part  of  the  way  the  Air  Force 
conducts  business.  Current 
use  of  M&S  by  Department 
of  Defense  (DoD)  program  and  prod¬ 
uct  managers  extends  throughout  the 
Air  Force;  from  research,  development, 
acquisition,  and  sustainment,  to  train¬ 
ing  and  operations  (Figure  1). 

The  New  fl&S  Vision 

The  Air  Force  envisions  an  integrated, 
common  M&S  environment  that  will 
be  accessed  by  analysts,  warfighters, 
developers,  and  testers  supporting  the 
range  of  Air  Force  tasks,  from  deter¬ 
mining  requirements  through  con¬ 
ducting  operations.  This  article  sum¬ 
marizes  trends  in  the  new  vision  for 
M&S  and  in  the  simulation  technolo¬ 
gy  that  can  be  employed  to  implement 
simulation  systems  of  the  future.  Joint 
M&S  standards  will  provide  key 
advanced  technologies  for  future  simu¬ 
lation  applications. 

Throughout  the  rest  of  the  decade,  the 
use  of  M&S  will  increase  throughout 
all  functional  areas  in  the  DoD. 
Because  of  increased  technical  capabil¬ 
ity  and  increased  fiscal  constraint, 
including  DoD-mandated  budget 
reductions  in  other  areas,  M&S  utiliza¬ 
tion  will  continue  to  expand.  Further, 
M&S  allows  DoD  organizations  to  do 
things  that  would  otherwise  be  unaf¬ 
fordable  (i.e.,  thousands  of  parametric 
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sensitivity  tests  on  new  systems)  or 
physically  difficult-to-accomplish  mili¬ 
tary  worth  studies  on  proposed  force 
structures  against  threat  command 
and  control  systems). 

Recognizing  the  importance  of  M&S, 
the  Department  issued  a  DoD  Direc¬ 
tive  on  “DoD  Modeling  and  Simula¬ 
tion  Management,”  that  provides  for  a 
DoD  M&S  Master  Plan.  As  part  of  the 
Master  Plan,  DoD  established  a  com¬ 
mon,  High  Level  Simulation  Architec¬ 
ture  to  assure  not  only  the  appropriate 
interoperability  of  simulations,  but 
their  interface  with  command,  control, 
communications,  computers,  and 
intelligence  (C4 1)  systems.  The  goals  of 


the  High  Level  Architecture  (HLA) 
include  several  areas: 

•  Interoperability 

•  Reuse 

•  Portability 

•  Distributed  Operation 

•  Legacy  Operation 

•  Scalability 

•  Broad  Applicability 

•  Technological  Evolvability 

•  Commercial  Off-the-Shelf  (COTS) 
Products 

•  Government  Off-the-Shelf  (GOTS) 
Products 

DoD  adopted  the  last  two  goals  as  part 
of  its  acquisition  reform  strategy  to 
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make  maximum  feasible  use  of  off-the- 
shelf  products. 

Today’s  simulations  are  narrowly 
focused,  stovepiped  developments  for 
each  user  community  Specifically,  they 
do  not  fully  meet  Joint  needs;  take  too 
long  to  build;  cost  too  much  to  build 
and  operate;  lack  verification,  valida¬ 
tion,  and  accreditation;  are  not  inter¬ 
operable  with  each  other’s  M&S 
assets;  and  are  not  easily  maintainable 
or  extensible.  High-level  DoD  and  Air 
Force  senior  acquisition  managers 
share  a  consensus  view  on  the  need  to 
interoperate  and  reuse  models,  simula¬ 
tions,  and  related  products  across  Ser¬ 
vice  lines;  across  traditional  communi¬ 
ties  (e.g.,  linking  models  and 
simulations  to  C41  systems);  across 
functions  (e.g.,  sharing  capabilities 
between  operations  and  acquisition); 
and  across  classes  of  models  and  sim¬ 
ulations  (e.g.,  linking  live,  virtual,  and 
constructive  simulations). 

The  effective  use  of  models  and  simu¬ 
lations  across  DoD  requires  a  com¬ 
mon  technical  framework  for  M&S  to 
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ensure  interoperability  and  reuse. 
Embodied  in  this  technical  framework 
will  be  a  common  HLA  to  which  mod¬ 
els  and  simulations  must  conform; 
conceptual  models  of  the  mission 
space  to  provide  a  basis  for  the  devel¬ 
opment  of  consistent  and  authoritative 
simulation  representations;  and  data 
standards  to  provide  common  repre¬ 
sentations  of  data  across  models,  sim¬ 
ulations,  and  C4I  systems. 

Air  Force  program  and  product  man¬ 
agers  are  in  general  agreement  that  no 
single  model  or  simulation  system  can 
satisfy  all  uses  and  users.  Further  defi¬ 
nition  and  detailed  implementation  of 
the  specific  simulation  system  archi¬ 
tectures,  which  will  be  HLA-compli- 
ant,  will  remain  the  responsibility  of 
the  developing  Service  or  Agency.  The 
HLA  will  specify  only  the  minimum 
definition  required  to  facilitate  interop¬ 
erability  and  reuse.  The  DoD  HLA  is 
central  to  the  M&S  Master  Plan. 

One  way  to  view  this  simulation  HLA 
is  to  think  of  a  city  planner  or  archi¬ 
tect.  A  building  is  compliant  as  long  as 
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Figure  2.  Joint  Synthetic  Battlespace 
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Figure  3.  The  Air  Force  H&S  Architecture 


you  get  the  right  permits  and  follow  the 
building  codes  and  standards.  Similarly, 
new  models  would  be  required  to  fol¬ 
low  specific  standards  to  fit  within  a 
certain  general  architecture.  The  DoD 
M&S  Master  Plan  and  subsequent 
DoD  directives  require  a  review  and 
oversight  of  all  ongoing  DoD  M&S  pro¬ 
jects  and  programs  for  compliance  with 
the  HLA  and  phase-out  of  non-compli- 
ant  programs  by  FY  01. 

A  New  Vector  for  Air  Force  N&S 

Consistent  with  the  DoD  vision,  the 
Air  Force  envisions  an  integrated, 
common  M&S  environment  accessed 
by  analysts,  warfighters,  developers, 
and  testers;  and  supporting  the  range 
of  Air  Force  tasks,  from  determining 
requirements  through  conducting 
operations.  On  June  9,  1995,  the  Air 
Force  convened  an  Air  Force  Four  Star 
M&S  Summit  to  create  an  M&S 
roadmap.  The  resultant  roadmap 
defines  a  future  vision  for  Air  Force 
simulation  and  describes  near-term 
and  mid-term  goals.  Achievement  of 
those  goals  is  expected  to  move  the  Air 
Force  closer  to  M&S  commonality; 
and  also  a  consistent  representation  of 
aerospace  forces  for  Joint  use. 

The  key  concept  in  the  Air  Force  M&S 
vision  is  the  Joint  Synthetic  Battlespace 
—  an  integrated  M&S  environment, 
connecting  analysis  and  training  and 


tying  together  many  types  of  simula¬ 
tion  (Figure  2).  The  simulations 
extend  from  high-level  aggregate  mod¬ 
els  to  detailed  engineering  models; 
from  pilots  in  live  aircraft  and  simula¬ 
tors,  to  hardware  components  and  lab¬ 
oratory  test  beds. 

The  Air  Force  M&S  infrastructure 
focuses  on  three  key  initiatives: 

•  Joint  M&S  Integration  Program 
(JMSIP)  —  a  coordinated  approach 
to  improving  air  and  space  represen¬ 
tation  in  our  legacy  models  and 
simulations  while  consolidating  into 
fewer  models  that  meet  the  require¬ 
ments  of  many. 

•  Joint  Standards  —  a  commitment  to 
Joint  M&S  developments  with  sup¬ 
porting  Air  Force  initiatives. 

•  Advanced  Distributed  Simulation 
Leveraging  -  programs  to  provide 
high-speed  connectivity  between  Air 
Force  installations,  multiple  net¬ 
worked  air  combat  training  simula¬ 
tors  for  each  wing  in  the  Air  Force, 
and  a  synthetic  battlespace  for  Joint 
Force  Air  Component  Commanders. 

In  the  near-term,  JMSIP  will  focus  on 
the  need  to  corporately  address  M&S 
improvements  and  the  need  to  encour¬ 
age  consolidation.  Addressing  these 
two  vital  needs  will  serve  as  a  leverag¬ 
ing  effort,  producing  an  Air  Force 


M&S  Roadmap  that  maximizes  com¬ 
mon  efforts  and  targets  improvements 
based  on  a  corporate  assessment  of 
their  importance  and  urgency 

For  the  mid-term  and  in  accordance 
with  overall  DoD  direction,  the  Air 
Force  will  implement  simulation  stan¬ 
dards  through  defined  architectures 
and  simulation  systems  that  support 
them.  Each  product  center  has  or  is 
developing  a  portal  into  the  Joint  Syn¬ 
thetic  Battlespace  of  the  future  for  sys¬ 
tem  of  systems  evaluations  and  a  key 
part  of  the  current  Air  Force  M&S 
infrastructure  —  Aeronautical  Systems 
Simulation  Analysis  Facility  (SimAF), 
Electronic  Systems  Command  and 
Control  Unified  Battle  Environment, 
and  Space  and  Missile  Center’s  Deci¬ 
sion  Software  Laboratory 

In  addition  to  key  facilities,  M&S  stan¬ 
dards  will  generate  greatly  improved 
simulation  interoperability,  allowing 
the  Air  Force  to  leverage  simulation 
investments.  The  Air  Force  has  target¬ 
ed  three  major  simulation  standards 
efforts  in  the  roadmap  for  high-level 
Air  Force  oversight  and  investment.  All 
will  participate  and  adhere  to  the  DoD 
High  Level  Simulation  Architecture 
initiatives  being  directed  by  the  Direc¬ 
tor,  Defense  Research  and  Engineer¬ 
ing,  and  managed  by  the  Defense 
Modeling  and  Simulation  Office: 
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•  The  Joint  Simulation  System  (JSIMS) 
is  a  distributed,  object-oriented  sim¬ 
ulation  architecture  and  system 
focused  on  the  operational  level  oj 
war  (campaign  and  mission  level 
simulation). 

•  The  Joint  Warfare  Simulation 
(JWARS)  focuses  on  Joint  campaign 
analysis. 

•  The  Joint  Modeling  and  Simulation 
System  (JMASS)  is  an  Air  Force- 
directed  program  to  develop  and 
deliver  a  distributed,  object-oriented 
simulation  architecture  and  system 
focused  on  the  tactical  level  oj  war 
(mission  and  engagement  simula¬ 
tions). 

These  Joint  standards  and  the  systems 
that  support  them  will  enable  interop¬ 
erability  and  reusability  of  Air  Force 
M&S  tools  across  key  communities 
and  processes.  The  Joint  standards 
serve  as  GOTS  frameworks  for  the 
addition  of  third-party  applications. 
These  initiatives,  coupled  with  ongo¬ 
ing  improvements  and  standards,  will 
bring  the  Air  Force  measurably  closer 
to  the  objective  of  a  common,  integrat¬ 
ed  M&S  system  (Figure  3). 

Computer  and  Simulation 
Technology  Trends 

The  changes  reported  in  this  article 
and  resultant  revision  in  the  DoD  and 
Air  Force  M&S  visions,  motivated  by 
changes  in  computer  and  simulation 
technology,  reflect  current  trends 
throughout  the  DoD.  In  the  past 
decade,  computer  hardware  technolo¬ 
gy  improved  several  orders  of  magni¬ 
tude:  microprocessor  speed  alone 
increased  about  100-fold.  The  over¬ 
whelming  trend  is  faster,  smaller,  and 
cheaper.  This  reduction  in  cost  and 
size,  coupled  with  an  increase  in 
speed  and  capacity,  resulted  in  a  mas¬ 
sive  increase  in  simulation  capability 
Computational  power  continues  to 
increase  as  prices  decrease. 

As  the  decade  moves  on,  a  multiproces¬ 
sor  on  the  desktop  will  be  common¬ 
place  for  simulation  and  analysis.  It 
will  be  accompanied  by  the  continued 
decentralization  away  from  the  central 
site  to  distributed  computing  personal 


processors  close  to  the  user,  mixed 
with  computationally  intensive  servers 
on  a  heterogeneous  network. 

Object-oriented  (OO)  software  tech¬ 
nology  is  having  a  major  impact  on 
simulation  technology  as  well  as  soft¬ 
ware  in  general.  For  software  develop¬ 
ers,  OO  software  addresses  three 
major  problems:  iterative  development, 
reuse,  and  maintenance.  Since  up¬ 
front  requirements  definition  is  diffi¬ 
cult,  many  successful  OO  projects 
employed  an  evolutionary,  iterative 
process  for  development.  Object-orien¬ 
tation  can  also  promote  reuse  through 
a  library  of  reusable  objects.  When 
combined  with  reuse  and  visual  pro¬ 
gramming,  OO  technology  can 
increase  productivity,  and  therefore 
lower  cost  and  decrease  time  for  soft¬ 
ware  development. 


Software  development  has  been  histor¬ 
ically  labor-intensive.  To  date,  even 
computer  aided  software  engineering 
tools  have  not  dramatically  increased 
productivity.  Producing  the  needed 
improvement  will  require  a  major  par¬ 
adigm  shift. 

OO  technologies,  combined  with  visu¬ 
al  approaches  and  an  engineering  dis¬ 
cipline  to  software  development  via  a 
software  structural  model  methodolo¬ 
gy,  can  finally  bring  the  needed  break¬ 
through.  OO  technology  will  allow 
implementation  of  component-based 
software  as  the  construct  for  software 
reuse.  By  employing  component-based 
design,  users  can  be  divided  into  four 
roles: 

•  Appliers  —  configure  input  data 
and  execute  existing  simulations. 

•  Assemblers  —  establish  connections 
among  component  parts  found  in  a 
reuse  library  to  build  simple  custom 
applications  or  models  without  pro¬ 
fessional  programming  assistance. 

•  Power  Assemblers  —  go  beyond 
piecing  component  parts  together 
by  implementing  more  complex 
logic. 

•  Fabricators  —  build  new  compo¬ 
nent  parts 

Advanced  User  Interfaces  will  extend 
the  now  common  Graphical  User 
Interface  into  an  agent-based  multi¬ 
sensor  user  interface  that  will  incorpo¬ 
rate  features  such  as  voice  synthesis 
and  voice  recognition.  Future  comput¬ 
er  software  architectures  will  incorpo¬ 
rate  Manager-Agent  and  Remote 
Programming.  In  Manager-Agent  pro¬ 
gramming,  the  client  computer  sends 
an  object  that  the  server  executes.  The 
object  is  called  an  agent  because  it  acts 
on  behalf  of  the  sending  computer.  In 
Remote  Programming,  the  client  and 
server  can  interact  independently  of 
the  network  once  the  network  trans¬ 
ports  the  agent  to  the  server.  These 
intelligent  agents  act  like  assistants 
rather  than  tools:  they  will  show  more 
initiative,  assume  responsibility  for 
larger  subtasks,  and  take  appropriate 
risks  (rather  than  confirming  every 
detail  with  the  user). 
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As  computer  and  software  technologies 
advance,  they  change  the  face  of  mod¬ 
eling  and  simulation.  Simulation  tech¬ 
nology  has  evolved  from  stand-alone 
models,  to  model  hierarchies,  to  an 
integrated  modeling  system  (Figure  4). 

Future  advanced  modeling  systems  will 
include  the  following  characteristics: 

•  Open  systems  architecture  support¬ 
ing  applications  conforming  to  com¬ 
mercial  and  industry  standards. 

•  Visual  paradigm  —  visual  program¬ 
ming,  visual  assembly,  visualization 
of  output  results. 

•  Object-based  to  allow  component 
reuse. 

•  Extensible  architecture  for  future 
software  concepts. 

•  Web-based,  browser-type  user  inter¬ 
face  on  the  desktop. 

•  Execution  on  distributed  heteroge¬ 
neous  network  of  workstations  and 
upscale  PCs. 

•  Tools  to  support  development  of 
model  components. 

•  Multiple  language  support  —  the 
user  can  specif/  the  target  source 
language  (C,  C++,  Objective  C,  Java, 
Ada83,  Ada95,  VHDL,  etc.). 

•  Object-oriented  database. 

•  Tools  and  models  support  a  “Plug 
and  Play”  concept. 

•  Supports  “distributed  model  devel¬ 
opment”  by  the  domain  experts  as 
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opposed  to  central  model  develop¬ 
ment  by  software  experts. 

•  Provide  a  repository  of  models  and 
their  components. 

•  Documentation  designed  to  support 
software  reuse. 

•  Verification,  Validation,  and  Accredi¬ 
tation  (W&A)  integral  to  the  soft¬ 
ware  development. 

•  Compliant  with  the  DoD  High  Level 
M&S  Architecture. 

Summary 

The  future  vision  for  Air  Force  simu¬ 
lation  is  a  flexible,  integrated  simula¬ 
tion  environment  that  supports  the 
full  range  of  Air  Force  activities. 
Revolutionary  and  evolutionary 
advances  in  computer  and  software 
technology  provide  significant 
opportunities  to  implement  this 
modeling  and  simulation  vision.  The 
new  M&S  technologies  will  permit 
the  creation  of  simulations  tailored 
to  the  user's  need,  at  a  greatly 
reduced  cost  in  time  and  money,  and 
with  elements  of  proven  quality. 
Admittedly,  achieving  the  simulation 
vision  will  require  patience,  perse¬ 
verance,  and  significant  investment 
to  overcome  many  challenging  prob¬ 
lems,  but  the  potential  payoff  is 
extremely  high. 
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Figure  4.  Evolution  of  Simulation  Technology 
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